Airbnb Forgoes SQL In Favor Of Memcached

Chris CrumDeveloper & Design

Share this Post

Airbnb recently shared an engineer Q&A talking about building its Airbnb Neighborhoods feature. Here's a look at the feature if you're unfamiliar:

Introducing Airbnb Neighborhoods from Airbnb on Vimeo.

Basically, it shows users pages about neighborhoods in cities, so they can decide where they want to stay when they visit.

You can read the full Q&A here, but here's the part where they explain why they ultimately went with Memcached over a SQL database or DynamoDB:

At first, it seemed we wanted a SQL database, as our data had relations. However, this was ruled out based on the need for mass updates. Next, we looked at an in-house NoSQL solution that we call Dyson. Dyson seemed to give us the flexibility we needed with writes and updates, so we tried it. For reference, Dyson is backed by Amazon's DynamoDB, a reliable, but limited, managed, NoSQL solution. In essence, if we put the data right into DynamoDB, then Dyson can serve it. This led to the creation of a DynamoDB cascading tap. Countless timeouts, headaches and late nights later, we had a working solution.

However, there was a problem, namely DynamoDB's 65KB storage limit. When you're storing uncompressed JSON, that's a pretty easy target to reach. As a band-aid, we engineered a solution involving pages of tuples. To say this solution was sub-optimal is putting it mildly, and the performance was even worse.

With launch quickly approaching, brilliant words saved the day: "You don't need a database, you need a [expletive deleted] cache" 1. So that's what we did, we traded our database for a cache. Specifically, we switched from Dyson to Memcached.

Here's a look at the Neighborhoods admin tool:

neighborhoods-admin-tool from AirbnbNerds on Vimeo.

[via GigaOm]
Chris Crum
Chris Crum has been a part of the WebProNews team and the iEntry Network of B2B Publications since 2003. Follow Chris on Twitter, on StumbleUpon, on Pinterest and/or on Google: +Chris Crum.