We caught up with Matthew Prince, CEO and Founder of CloudFlare, at London Web Summit a couple of weeks back. In this in-depth interview, he tells us about CloudFlare, and how it came about – including his hopes that he can build a service that will make things like denial of service attacks a thing of the past that we’ll study in history books. He also went in to some detail about how he started, sharing some of his experiences on how more startups should be going for the big ideas – he told us how ‘…assembling resources for a small project may seem easier at first but in the end – that project ends up being harder to attract developers for, to really scale it to attract customers for…’ ‘…and so focusing on big ideas paradoxically becomes easier than focusing on small ones’.
He also told us about some challenges which he faced, as well as some of the interesting trends he’s witnessed at CloudFlare – including the ‘…whole bunch of surrogate mother sites come under attack by what appeared to be an organisation in China that was opposing surrogate motherhood’. ‘The day before Valentine’s Day, February 13th, we saw a number of small-business flower shops come under attack: a purely financially-motivated essentially extortion scheme that said pay us a thousand dollars or we’ll knock your site offline on the busiest day‘.
The audio is available in the player below, and the interview is transcribed in it’s entirety below.
Two hundred and fifty million websites out there really never had a very good security solution
How did the idea for CloudFlare come about? What led to its inception?
There were three of us who started CloudFlare: myself, Michelle Zatlyn, and Lee Holloway. It was started with the idea that the top ten thousand websites that are online have a lot of solutions to make sure that their site is fast and make sure that their site is safe, but the other two hundred and fifty million websites that are out there really never had a very good solution. We wanted to build something that made the internet faster and protected websites from bad guys, and we wanted to make it easy and affordable for literally anyone with a website. That was the original mission of CloudFlare, and we’re proud that that mission has resonated and we’ve grown as much as we have over the last two years.
How does CloudFlare work?
CloudFlare makes the internet faster and protects websites from attackers and bad guys. Previous solutions had required you to install hardware or software, or change code on your site. What we did that’s a little bit different is exist inside the network itself. So you can think about it that if the network is a series of tubes that are all connecting various websites together, we make sure that the traffic that flows through those tubes flows through our network. We run data centres in fourteen different locations all around the world and traffic passes through those facilities, and when it does we can make sure that any of that traffic isn’t some sort of an attacker or threat, and then we can also compress the traffic and do a lot of things in order to make it significantly faster. On average we’ll stop a wide range of attacks ranging from simple things like spam all the way up to real threats like denial of service attacks, and we’ll also make sure that an average website is twice as fast anywhere someone’s visiting it, around the world.
How much does CloudFlare speed up websites?
We do a lot of different things in order to speed up sites. On average we’ll make a site twice as fast for it to load. It’s fast enough that you can feel the difference.
There are four major things that we do:
- The first – which is probably the easiest to understand – is that we’ll take parts of the website which are static, that don’t change. So imagine the logo on a web page or maybe an image which doesn’t get updated very often, and we’ll detect that that’s a static part of the site and we’ll move that content closer to the person actually visiting the site. The speed of light is very, very fast but it still takes almost half a second for a photon of light to travel from Europe to San Francisco.
- If we can make sure that your images are actually closer to your actual visitors then that increases the performance of the loading of the site, and makes sure that there’s less load on the actual web server.
- We also do things like significant compression and if you can make a file half as big it’s as good as making the line twice as fast. We decrease the overall load on a web server.
- The last thing we do is we’ll actually do in-page tuning for whatever device is behind it. So you can imagine if somebody comes to your site and they’re on an iPhone, we deliver a slightly different version of the page optimised to render for that device. So it won’t change the look and feel of the page but it’ll actually make it significantly faster.
All of these things add up to, on average, doubling the performance of any website that’s behind our network.
If you mess with one bean, you mess with the whole burrito
From a security perspective, how much stronger does CloudFlare get on a weekly or even daily basis?
One of the key things that was a mission of CloudFlare was to make sure that every time one part of the site was attacked or one customer of ours was attacked, that the knowledge about that attack would spread to all of the other sites. One of our customers wrote in the other day and he said that the tagline for CloudFlare should be ‘If you mess with one bean, you mess with the whole burrito’ and I kinda liked that, I thought that was funny. But the system is constantly learning, so literally every second at each data centre new rules are being propagated based on the traffic that is coming through that. CloudFlare sees a ton of traffic, we see more traffic through our network than Amazon, Wikipedia, Twitter, Zinga, AOL, Bing and Apple combined, and what we can do with all of that information is make intelligent choices about whether someone coming to a site is a threat or whether they’re a good guy. If they’re a good guy we want to make sure they get there as fast as possible, and if they’re a threat we want to stop them. So every single day the system gets significantly smarter and is learning about new attacks, and that’s why it’s powerful that the system has so many different sites that are currently using it.
We’ve had to face challenges like ‘how do you get a server through German customs?’
You’re coined as the fastest-growing start-up in history, using certain metrics. How did you manage to scale so efficiently and what were the biggest challenge?
We had a marketing plan and we had ideas of having sales forces and things; the challenge has been that fifteen hundred new websites sign up for CloudFlare every single day, and so building out ahead of that growth has been difficult. We have a great team of engineers that have worked extremely hard in order to make sure that we’ve got the resources necessary in order to build that out and we’ve had to learn a lot from very technical systems and how you scale those, all the way to challenges like ‘how do you get a server through German customs’. All of that is going back into the product and allowing us to stay ahead of the growth, but it’s definitely a challenge to do that.
Is using a shared pool of resources a bad thing?
Some commentators say that you’re effectively sharing all of your resources with all of the other sites that you’re acting as a proxy for. It’s a shared pool of resources. What do you make of this criticism?
Philosophically, one of the challenges that we’ve always had with the security industry is that information tends to get siloed within an organisation. If you imagine if Google is attacked and Yahoo calls up and says ‘hey, can you share some information about that attack so that we can protect ourselves?’, that information doesn’t get shared from one organisation to another. So when we started CloudFlare what we said was ‘let’s find a way that we can actually share information across a bunch of sites’. So we’ll never disclose this particular site was under attack, but we will use the information about that in order to protect other sites. That obviously is a real threat to some of the security industry, which has traditionally found that selling their services at an extremely high price to a small number of clients is a good business. We’ve turned that on our head and said ‘let’s try and make this available to anyone online’, because we don’t believe you should have to have a lot of money in order to have a fast and safe internet site
We’ve architected the system in order to avoid a single point of failure
Clearly centralisation in security is a good thing. On a technical basis, what happens, for instance, if your DNS systems fail? Is there a single point of failure? Do you think your customers have to have a high level of trust in your service? What kind of assurances can you make to customers who have these concerns?
In terms of a single point of failure, we’ve actually architected the system in order to avoid that to the extent possible. We currently have fourteen data centres around the world and we’re continuously adding more, and we’ve designed the system so that if any server in any one of those data centres has a problem, or even if the entire data centre itself goes offline, the traffic just automatically flows to the next closest data centre. We know that we’re not perfect, no service is, but we’ve tried to make sure that in the architecture of the system it’s designed to expect that there are going to be problems and then be resilient around those issues. In the entire time since we’ve been public – for the last eighteen months – we’ve only had one significant period of outage and that was for about a five minute time period, and our team corrected it very quickly and got everyone back online.
We work hard to build the trust of our customers and you need to trust any vendor that you’re working with. One thing that is helpful in policing that trust is that it only takes five minutes to sign up for CloudFlare, but if you’re ever unhappy it’s only two minutes to switch back off the system. We track that rate and we know when we’re doing well when our churn rates of people leaving fall. We have lower churn rates than mobile phone companies that have to lock you in with contacts, we never have contracts or anything like that, so we’re pretty happy about that.
Another criticism is the issue of false positives and blocking users from sites. How confident are you that legitimate traffic is not blocked? Could blocking potential customers and scaring them with quite an intimidating page be detrimental to some online businesses?
We believe that this is one of the most important things for any security company, is to make sure that you’re letting the good guys in and not letting the bad guys in. The challenge is that the bad guys are often using the resources of the good guys, so the business model of a virus writer is to write a virus to infect a lot of computers and then use those infected computers that are part of what’s called ‘the bot net’ in order to launch attacks against other sites. Unfortunately it’s sometimes hard to tell what’s a virus that’s launching the attack versus when it’s a legitimate human just coming in, and so what we’ve done is we’ve allowed site owners, the publishers that use our network, in order to choose the level of security that they want so if they’re highly concerned with security they can turn the security level up.
The trade off is that you’re more likely to have false positives
The trade off then is that you’re more likely to have false positives, and if you’re less concerned about security then you can turn that security level down in which case there are almost zero false positives through that. So we believe that the right thing to do is to allow that choice to be the publishers, to allow them to make a choice about what level of security they want and then help inform them about the trade-offs in either of those cases. Whenever we do challenge someone when they’re trying to come to the site we try to give them an opportunity first, to see what the reason it was that they were challenged. Like, ‘your computer’s infected with a virus’ or something like that. Give them the tools that they need to protect themselves and then give them the opportunity to prove that they are actually a good guy, a human being – by filling in the squiggly letters in a box, what’s known as a Captcha, or doing something like that. That information all gets fed back into the system and so every time one of those Captchas is passed it increases the reputation, it makes the reputation more good for that particular visitor. This ensures that they’re never going to see that type of a challenge again, but that’s something that we’ve spent a lot of time trying to refine. Again, what we believe is important is that publishers should be able to set their security level based on what their tolerance is for risk, and their tolerance for false positives.
It sounds ridiculous, but our goal is to build a service that literally powers the internet
CloudFlare sounds like a complete no-brainer. Is there any situation that you would recommend not using it, or is there any website or service online that shouldn’t use it? And if there isn’t, could you potentially scale to every website in the world?
It sounds ridiculous, but our goal is to build a service that literally powers the internet and we’re on track toward that goal. We’re adding more than fifteen hundred new websites a day and that rate of growth is only accelerating. The sites that don’t make sense for CloudFlare are usually sites that are doing something particular that don’t work very nicely with our system. So, a site that is streaming content directly from the site itself, so if you’re doing video streaming or something like that then our system isn’t set up to do that. That’s actually much more of a job for a traditional CDN or content delivery network, and so we often will refer customers to Akamai or Limelight or some of the traditional CDN players if they’re doing that. But the number of sites that are actually streaming videos is relatively small, so if you have a site and you embed a YouTube video or you embed another service in it CloudFlare will continue to work just fine.
YouTube, on the other hand, would not be a particularly good customer for us today, but there aren’t many focused that way. But our goal is to continue to learn and get smarter with each new site which joins the network, and as a result of that we hope that we can build a service that will make things like denial of service attacks a thing of the past that we’ll study in history books. Because together we can pool resources in order to make sure that every site online is fast and safe.
“Big ideas are easier than small ones”
“Big ideas are easier than small ones”, and it’s easier to inspire people to work around you if you have a big idea. What do you think stops people having these big bold ideas? Is it the fear or is it something else?
There’s a guy named Paul Graham who started an organisation called Y Combinator who recently wrote an essay about really big ideas that VCs would be excited to fund, but just reading them will scare away entrepreneurs. If you think about it, some of the big revolutionary technology ideas that have happened there’s no way that when the founders started it they could have all the resources or know everything that they needed to in order to actually build what they thought.
When we started CloudFlare we honestly had no idea what we were getting into, and had we really known it probably would’ve scared us quite a bit because we’ve had to learn about everything, from the way IP addresses are calculated in Latin America to how to deal with law enforcement agencies when they contact you. There’s a lot of stuff that you had to do in order to do this and we’ve learnt that at every step along the way.
Often times when entrepreneurs think about starting businesses they look around and they see what resources they have immediately at hand and think of that as the finish line
Often times when entrepreneurs think about starting businesses they look around and they see what resources they have immediately at hand and think of that as the finish line, as opposed to thinking of that as the starting point from which they can jump off and continue to assemble resources to build to a much bigger point. Assembling resources for a small project may seem easier at first but in the end that project ends up being harder to attract developers for, to really scale it to attract customers for. Focusing on big ideas paradoxically becomes easier than focusing on small ones.
When hiring, even if there is a gap in experience between two hires, you have a hire that comes to you who really wants the job – like you were saying, there’s people sleeping in the corridor waiting to work for you – and they may not have as much experience as someone you’re going out of your way to recruit, which one is the better hire?
There’s a time and a place for both, but we definitely prefer the former to the latter.
Most of our employees at CloudFlare were customers before they were employees. One of the great things about having a service which is easy for anyone with a website to afford and use means that we have a lot of users that become passionate about the product, and then approach us and explain why they would be then good employees for us. We look all over the place for employees and we’ve had some employees that we’ve actively recruited, we’ve recruited people from Google and Yahoo and Facebook and a number of places, that have come to CloudFlare in order to work and in some cases it’s been people who’ve come directly to us.
I think that’s why a lot of very talented, very, very interesting engineers are joining our team.
What we think about recruiting more than anything is that we look for people who are good at working on teams, and who we want to spend a lot of time with. So we’re much more focused on the personality of the candidate than necessarily a particular skill set. ‘Cause if you’re a nice guy or gal who’s curious and likes to learn and wants to solve hard challenges, then that’s going to be a much more successful hire over time. It’s pretty cool to work at CloudFlare, there are very few places on Earth that if you’re an engineer you can push a line of code and over the course of the next twenty-four hours you’ll effect over a hundred million people. That’s sort of the state that we’re in right now.
Say no often – we listen to our customers carefully, we pay attention to them, we just don’t let them necessarily dictate our product development
Another one of the issues you raised was the ‘say no often’ part of things. When is a customer-driven organisation the right thing to do? And when is it the wrong thing to do, and how hard is it to resist that customer?
We listen to our customers carefully and we pay attention to them, we just don’t let them necessarily dictate our product development. When I said in the talk earlier today that it’s important to say no, it’s really hard when a big Fortune 500 company comes to you and says ‘we’ll pay you twenty/thirty/forty thousand dollars a month if you’ll just make the following changes’. What we’ve found is that we put those change requests on the board, and we evaluate them not next to the fact that they’re worth twenty-thirty-forty thousand dollars to us but next to ‘is this something, which is moving the product forward’. Usually if the changes really make sense then they’ll get made at some point and at that point we’ll go back to those customers, but what we don’t want to do is end up having a few customers who dictate what it is that we’re working on. We want to be driven by the smart engineers that work for us, and they’re designing a product to scale to power the entire internet as opposed to having a small handful of customers that are really dragging us around and telling us what to do.
Under-price and over-simplify - focus on ‘viable’ in the ‘minimum viable product’
“Under-price and over-simplify” was another one of your philosophies. Are you sceptical about the MVP philosophy and how do you focus on what’s importan
I think that the ideas of a minimum viable product, the MVP philosophy, are good but a lot of times people don’t spend the time with their first products in order to get them in ship-shape to really launch. What concerns me is that you only, as a start-up, get a very short window of time and attention in order to actually get out and make a big splash and grow. Our secret advantage was that we had a large community before we launched publically that trusted us and was willing to let us experiment and really get the product right before we turned it on and launched it. That’s the best case. If you can quietly test behind the scenes before you reach out then that makes a lot of sense, but as soon as you go live the expectations of the quality of web service are so high today that you’ve gotta get that right from the beginning. I think that the idea of minimum viable products is right, but when you’re ready to really launch your start-up make sure that you really focus on ‘viable’ in the ‘minimum viable product’ sense; get it out there, make sure it really solves a problem and then the way to keep that under control is to simplify the product down to its core essence and make sure that that core essence is as good as possible, and if you do that customers will really respond.
We saw a whole bunch of surrogate mother sites come under attack by what appeared to be an organisation in China that was opposing surrogate motherhood
In your experience, what kind of sites are generally subjected to more hacking attacks?
It really ranges. We saw, for example, a whole bunch of surrogate mother sites come under attack by what appeared to be an organisation in China that was opposing surrogate motherhood. The day before Valentine’s Day, February 13th, we saw a number of small-business flower shops come under attack: a purely financially-motivated essentially extortion scheme that said ‘pay us a thousand dollars or we’ll knock your site offline on the busiest day’. We’ve witnessed in the last six months a six hundred percent increase in denial of service attacks launched at a wide range of sites, so it’s hard to point to any one particular type of site that’s there. But philosophically the internet’s one of the greatest inventions of human kind and what it stands for, the telos of the internet, is that anyone with an idea can make that idea available to a global audience. Something like a denial of service attack is really a form of censorship, it’s one person with power using that power in order to knock offline an idea that they don’t agree with or they don’t support. We think that what we’re trying to do is make that impossible, to allow the internet to achieve its full potential. Where anyone, regardless of how controversial the idea is, is able to keep it online.
One of the somewhat ironic things that happened fairly recently was we were getting attacked and had complaints launched against us by an Iranian group that was protesting the fact that a pro-Israeli website was on CloudFlare, and at exactly the same time we also had attacks and had protests from a bunch of Israelis that were protesting the fact that a pro-Iranian group was on us.
CloudFlare doesn’t see our mission as a censor, we don’t think that we know what the right thing for the internet is. We just believe that the internet is a tool to spread knowledge and let anyone have access, and we want to make sure that you don’t have to have a lot of money and you don’t have to have a lot of technical skill in order to ensure that you have a site which is fast and will stay online even if you’re attacked
“It’s a bit of a fool’s errand to come after us”
So as controversial and potentially immoral sites are generally subjected to more hacking attacks? Is there a moral dilemma in supporting organisations like Lulzsec? Are there legal implications – where do you draw the line? Is this almost a disaster waiting to happen?
I teach law, I’m a recovering lawyer, and so these are issues that I think about quite a bit. You’re correct in saying that we’re not the host of the site and you can’t actually launch a hacking attack through our service. The most you can do is publish information and so again – we’re a law-abiding organisation and if after due process a court in a jurisdiction that we operated in told us that we needed to stop providing service to some particular organisation or another, we would probably follow that requirement because again, we’re not hiding form the law and we’re not designed to hide from the law. But there are two things that are important: the first is, if that order ever came and we knocked someone off our service that wouldn’t take the content away, it would simply make it a little bit slower and make the content slightly more vulnerable to attack. So it’s a bit of a fool’s errand to come after us.
The second thing is, it’s actually pretty amazing that we’ve been around for eighteen months now publically, we have had organisations that ranged from hacking groups like Lulzsec all the way over to organisations that those hacking groups attacked and everything in-between and never in that entire time has any law enforcement officer ever asked us to remove content from our network. Whilst I’m not under the illusion that that day may come at some point, I think that I’ve been pleasantly surprised how clued in law enforcement is to the fact that we aren’t the host, that hacking can’t be launched from our network, and instead if they were to knock a service off of us it doesn’t make the content go away it would just make it a little bit slower to be published.
I’m not sure that anyone has elected me to a position where I should be deciding what can and cannot be published
How beneficial to your company, not just on a technical basis but also on a business basis, is having hacking organisations or Lulzsec using CloudFlare? What does this mean in real terms obviously to your network but also to your company’s reputation, to the press it receives? You received a lot of press as a result of the Lulzsec, did this play any part in your success?
That’s a hard question to answer – because I don’t have a time machine to go back and play it out the other way. I think from a purely business perspective an organisation like that is very unlikely to upgrade to a pro account and pay us, so it’s not – from a financial perspective – a particularly positive thing. But what is very positive is the technical knowledge that we gain as a result of servicing sites that are under attack. So the Lulz security site, for example, came under a fairly aggressive attack for the twenty-three days that they were on our service, and in that course of time our system automatically generated over a million new rules and those rules were then used to protect other sites that are on the system. The core value of CloudFlare is again that every single site on the system makes the server smarter and that was true in their case and other cases, but it’s also true with organisations that are on the other side of the spectrum – like various US government sites that use CloudFlare and also even governments in Turkey and Malaysia and Ghana and everywhere else in the world, but the internal revenue service of Pakistan uses CloudFlare in order to keep their site safe. So again we don’t discriminate based on content, largely because it’s just a slippery slope that doesn’t make sense for us to be making a choice on. We’ve been just very public about the fact that our goal is to power the internet, and there are a lot of things on the internet that I find troubling but I’m not sure that anyone has elected me to a position where I should be deciding what can and cannot be published.
You’re supporting US government sites and Turkish government sites, you’re working closely with governments. What do you make of cyber-warfare, governments playing against each other? Will you ever find yourself starting to get wrapped up in politics? How close can you get to politics without actually getting involved in it?
We’ve never actually talked to anyone in the US government. They sign up for our service the same way that anyone else does, which is that they went to the website, filled out some information and came in. If they called us up and said ‘hey, we’d like you to protect X, Y or Z large site, please fill out the following RFP and come play golf with us’ our answer would be what I said above, which was that that’s just not our style and we’re trying to build a service that is out there, so it’s not a problem that we’ve had, I can’t imagine any circumstance under which we would let any attack through regardless of who the client was. For the most part we don’t know who most of our clients are. The only things that you need to sign up for CloudFlare is an email address, a username and to make up a password. These are bridges which we’ll cross when we come to them, but I doubt that we come to them.
Do you see a time when you could license your system and do you think that if you were to license it, it would be attractive to those ‘older security companies’ that you’re disrupting?
So we work with a number of other companies in order to act as channels for our service now. Largely those are hosting providers and so I think that there may be opportunities to expand channel program like that, but the real value of what we have is that a bunch of people working together behind one product can create a better internet for all of them. That makes it challenging for competitors just to have the data that we do on what threats are, and so we’d be happy to work with just about anyone but right now I think we’re definitely leading this part of the industry.
In order for the internet to really survive and thrive, we believe two things have to happen
So to summarise: what are the main, broad, long-game core values of CloudFlare? Where do you envisage it in five years? Do you see any competition coming from any direction? You mentioned that you’re disrupting the security industry, do you think those industries are going to have to start innovating soon?
In order for the internet to really survive and thrive, we believe two things have to happen:
The first is that it has to become less expensive and easier for publishers to publish, and CloudFlare is one part of an ecosystem in order to do that. We hope to play an important part to help that part of the ecosystem thrive and ensure that again, anyone that has a good idea can put it online and reach a global audience with that idea and not have to worry about either attacks or scaling or anything like that. That we will be able to handle that for them. So that’s our vision on that side, and at the same time we believe that it’s important that anyway that publishers are making money online, that we should be able to help them make more. An easy way to think of this is that Amazon sees, for every tenth of a second it can shave of page load time, a one percent increase in revenue. That’s the start but over time we think that there are a lot of additional ways that we can actually help publishers earn more from their sites, and that’s important in order to make sure that there’s going to be high-quality publishing on the internet. I could never, in our wildest forecasts we’d have never thought that the last eighteen months would be like it has. It’s been very, very successful, we have an amazing team.
What the next twelve months, what the next five years brings I have no idea but we’re going to continue to work towards our mission, our goal, which is to make the internet faster and protect websites from bad guys. To eventually build a faster, safer, smarter internet that CloudFlare can continue to be a real part of. So it’s an exciting time for us and we’re really excited with what we’re working on.