Airbnb Forgoes SQL In Favor Of Memcached

Airbnb recently shared an engineer Q&A talking about building its Airbnb Neighborhoods feature. Here’s a look at the feature if you’re unfamiliar:

Basically, it shows users pages about neighborhoods in cities, so they can decide where they want to stay when they visit.

You can read the full Q&A here, but here’s the part where they explain why they ultimately went with Memcached over a SQL database or DynamoDB:

At first, it seemed we wanted a SQL database, as our data had relations. However, this was ruled out based on the need for mass updates. Next, we looked at an in-house NoSQL solution that we call Dyson. Dyson seemed to give us the flexibility we needed with writes and updates, so we tried it. For reference, Dyson is backed by Amazon’s DynamoDB, a reliable, but limited, managed, NoSQL solution. In essence, if we put the data right into DynamoDB, then Dyson can serve it. This led to the creation of a DynamoDB cascading tap. Countless timeouts, headaches and late nights later, we had a working solution.

However, there was a problem, namely DynamoDB’s 65KB storage limit. When you’re storing uncompressed JSON, that’s a pretty easy target to reach. As a band-aid, we engineered a solution involving pages of tuples. To say this solution was sub-optimal is putting it mildly, and the performance was even worse.

With launch quickly approaching, brilliant words saved the day: “You don’t need a database, you need a [expletive deleted] cache” 1. So that’s what we did, we traded our database for a cache. Specifically, we switched from Dyson to Memcached.

Here’s a look at the Neighborhoods admin tool: