Twitter Makes Its MySQL Work Open SourceBy: Zach Walton - April 10, 2012
MySQL is one of, if not, the best database management systems. It’s even better because it’s open source which allows anybody to make changes to it as they see fit for their own needs. Major companies use it to power their Web sites and it’s always nice to see these companies make their work with MySQL available to the public.
Twitter announced today on its engineering blog that its work with MySQL will now be available to all under an open source license. This is pretty big since Twitter is basically built on MySQL from the interest graph and timelines to user data and the tweets themselves.
Some of the work that Twitter has done with MySQL includes the following:
Add additional status variables, particularly from the internals of InnoDB. This allows us to monitor our systems more effectively and understand their behavior better when handling production workloads.
Optimize memory allocation on large NUMA systems: Allocate InnoDB’s buffer pool fully on startup, fail fast if memory is not available, ensure performance over time even when server is under memory pressure.
Reduce unnecessary work through improved server-side statement timeout support. This allows the server to proactively cancel queries that run longer than a millisecond-granularity timeout.
Export and restore InnoDB buffer pool in using a safe and lightweight method. This enables us to build tools to support rolling restarts of our services with minimal pain.
Optimize MySQL for SSD-based machines, including page-flushing behavior and reduction in writes to disk to improve lifespan.
It’s sure to be quite amazing to finally get a look at how Twitter uses MySQL. As Twitter themselves says, this can only help to foster innovation among those who use MySQL for their services. If you want to start getting a crack at how Twitter uses MySQL, check out the developer documentation.
As an extra bonus, the Twitter team will be speaking on Gizzard, their “sharing and replication framework on top of MySQL, at the Percona Live MySQL Conference and Expo on April 12 in Santa Clara, California.
As with all open source releases, you can get your mits on the code right now via github. There’s quite a lot here, but it’s well worth looking through. Who knows, you may even learn a thing or two.