Application vs Infrastructure Layers in Web 2.0 - A question of Scalability

This post was prompted by two recent ones, on GigaOm and Tara Hunt's blog. First GigaOm, in an article called Web 2.0 & Death of the Network Engineer written by Allan Leinwand

I was recently meeting with a Web 2.0 company discussing their network infrastructure plans. As I started asking questions about their racks of servers, their storage area network (SAN), their plans for routing, load-balancing and network security, the CTO of the company stopped me and made a bold statement.

He said, “The Internet is like electricity. We plug into it and all of the things that you mention are already there for us. We don’t spend any time at all on network or server infrastructure plans.”

To this CTO, knowing the details of his network and server infrastructure was like knowing the details of the local utility electricity grid – not required. Is this a bad thing, or proof that networking technologies have succeeded?

Next Tara Hunt, in a case study (Twitter) of using onramps:

Things were pretty slow those days and a few of us dedicated exhibitionists would send the occasional message, but mostly, as Narendra aptly put it, Ev was our Tamagotchi. Lucky for all of us, Ev has an insanely interesting life.

If things would have stayed that way, I hardly think Twitter would have lasted very long. Now we know, it didn’t and what they did next is the most interesting part of the story…as well as a lesson for people building web apps everywhere.

It is at this point that most companies make one of a couple of mistakes:

1. They throw in the towel, scrap what they are doing and build something entirely different, usually using the same technology. Riya did this. Goowy did this. This can be met with great success, of course, like in the example of Flickr’s directional change. But I would also caution that these types of changes mid-stream can be the lack of deeper thinking about the real problem.

[Lucky for us, Twitter was an interesting enough concept to realize it’s potential. It had a small following, but you could see the love already.]
2. They try to build in more features…more “reasons for people to stay”. I’ve seen this a number of times and it usually follows this conversation:

Bob: “We have high traffic, but nobody is sticking around.”

Alice: “I keep hearing the same thing. They just don’t have enough to do.”

Bob: “Well, then we should build in more things for people to do.”

Then the fortress continues to grow and, in consequence, it becomes less and less attractive for a person to sign up.

Twitter could have done this. They could have built in groups or additional short codes or created all sorts of ways for people to use their SMS. But they didn’t.

Instead, they followed a THIRD option: they built more onramps.

They added Jabber integration for sending/receiving messages, and added sending messages to the web interface. They also allowed you to keep track of just your friends in the web interface. They also released an API, allowing for others to build many more onramps.

The problem with the Onramp approach is the same issue as alluded to in the first post - ie scalability!

We've noticed this with Web 2.0 startups in general, they talk about applications and presentation without really understanding the infrastructure issues. As one of the commentators (Jake McKee) on Tara's blog puts it:

1. When an onramp shuts down due to technical problems, you lose users who have become dependent on those onramps. For instance, the only way I typically use Twitter is via the AIM bot. AIM bot is broken, no clue when they’re fixing it, so I’ve largely stopped using Twitter. Adding onramps without fully developing them to last, stay robust is no different than adding a bunch of new features, and then you’re back to your point about feature addition.

2. Onramps = usage = scalability issues. Twitter, at least to me, has basically suffered the Friendster Folly - successful to a point of a nearly uselessly slow site. I can’t believe this didn’t have something to do with the fact that there were so many onramps for users to submit overwhelming amounts of content.

Couldn't have said it better myself - so I didn't

We've also noticed this in chatting to entrepreneurs in the London scene (see our note here for example). We think they just don't know what they don't know. Unless they were there in Web 1.0, or have a Telco/ISP background (or were ever dugg) they probably have never seen the impact of poor scalability.

And its not as if the Amazon Grid or a Web Hoster / ASP will save them, it was made fairly clear in questions after Amazon CTO Werner Vogel's talk at FOWA 07 that even on the Amazon service there is a layer of user management (aka "knife and forking") required between any one application and the webservice infrastructure.

It was ever thus!!

No sooner has the "viral marketing / cashola / PR whatever" taken root, and the users beat a path to your door, then your service falters, sputters and falls over. Grumpy users go away, egg is on face, service reputation is shot and cost of getting back into the game is a lot higher than first time round.

So, what to do?

Well, start by getting help from people who know what they are doing - like us

But the most critical thing to do is just to realise that Scalability will be an issue, and one can't magic it away by assuming its like electricity. Once that is understood, then planning for it can begin.

(An Ad aside - In our previous lives we have been involved in the OSS and other infrastructures behind ISPs, ASP/web hosting, cable triple plays and "Web 1.0" services, and getting scaling wrong is what can kill a business completely. At Broadsight we have done quite a bit of work helping various "Web 1.0" and Cable / DTV companies transform into Web 2.0 - and scaling is always an issue)

But, I'd rather let one of the commenters on GigaOm close - Ed Byrne:

Coming from a company that hosts many web 2.0 and SaaS applications, I know all to well the CTO that thinks hosting is a utility like electricity.

I believe that statement is true at the low end - but not when hosting a serious application. An example that comes to mind is when a client wanted full redundancy in their architecture and it all fully load balanced. Sounds standard - we will spec and build. Of course our network engineering team asked all the usual questions - but when it came down to going live - we discovered the application needed layer 7 load balancing, not layer 4 - which our devices provide. A layer 7 load balancing is at least 4 times more expensive.

This is clearly a case where the client’s lack of understanding is going to have an impact on their deployment time as well as a financial impact.

The issue is further complicated if one is using higher bandwidth media, especially video, as the rate of bandwidth, disk, and processing scale can raise far more rapidly than just serving webpages will.

I can't reiterate this too strongly - anyone who thinks the Web of today is just like electricity is probably in for a nasty shock!