Facebook announced its 500 million user milestone yesterday, and along with that announced some other stats like how it gets over 100 billion hits per day. This was revealed in a post in which the company talked about how it scales with its massive growth.
Twitter is also growing rapidly, and the company is also talking scaling. This is of particular interest, given that Twitter frequently breaks, and offers up a "fail whale" (or not even that in some cases) instead of a working service.
"As we said last month, we are working on long-term solutions to make Twitter a more reliable and stable platform," says Twitter PR guy Matt Graves. "It’s our number one priority. The bulk of our engineering efforts are currently focused on this issue, and we have moved resources from other projects to focus on it."
Twitter Engineer Jean-Paul Cozzatti, who compares the tasks of scaling, maintaining, and tweaking Twitter to building a rocket in mid-flight, says the company has made over 50 optimizations and improvements to its platform. Among these are:
– Doubling the capacity of Twitter’s internal network
– Improving the monitoring of Twitter’s internal network
– Rebalancing the traffic on Twitter’s internal network to redistribute the load
– Doubling the throughput to the database that stores tweets
– Making a number of improvements to the way Twitter uses memcache, improving the speed of Twitter while reducing internal network traffic
– Improving page caching of the front and profile pages, reducing page load time by 80% for some of Twitter’s most popular pages
"While we’re continuously improving the performance, stability and scalability of our infrastructure and core services, there are still times when we run into problems unrelated to Twitter’s capacity," writes Cozzatti.
He’s referring to earlier this week, when Twitter experienced a database problem, which led to users not being able to sign-up, sign-in, or update their profile or background images. There were also problems with the API. He says Twitter has taken steps to detect and respond to such issues more quickly in the future.
To help make Twitter more reliable for the long haul, the company is getting a new data center that will give it more control. They are also looking to hire more engineers, and have 20 positions open.