Netflix Introduces Exhibitor, A Supervisor System For ZooKeeperBy: Zach Walton - April 17, 2012
It was announced last week that Netflix would be open sourcing its “monkeys” so people could get a look at how they handle their cloud infrastructure. It’s invaluable information considering that Netflix is one of the major companies to do a great deal of their business from the cloud. Keeping up with the animal motif, the company’s coordination service is called ZooKeeper and it’s going open source.
A part of ZooKeeper, Curator, had already gone open source. It was a client library for ZooKeeper, but Netflix saw a need for a supervisor service “that runs alongside ZooKeeper server instances.” This led to the creation of Exhibitor and it’s going open source from the start.
Netflix details the arduous effort that goes into currently managing ZooKeeper. A lot of it has to do with the fact that developers have to manually configure a ZooKeeper ensemble. This requires an ensemble to be “hard coded into a configuration file that must be identical on each ZooKeeper instance.”
That’s where Exhibitor comes in as a service that should make developer’s lives easier when using ZooKeeper. Here’s the list of features that you can expect when using Exhibitor:
Instance Monitoring: Each Exhibitor instance monitors the ZooKeeper server running on the same server. If ZooKeeper is not running (due to crash, etc.), Exhibitor will rewrite the zoo.cfg file and restart it.
Log Cleanup: In versions prior to ZooKeeper 3.4.x log file maintenance is necessary. Exhibitor will periodically do this maintenance.
Backup/Restore: Exhibitor can periodically backup the ZooKeeper transaction files. Once backed up, you can index any of these transaction files. Once indexed, you can search for individual transactions and “replay” them to restore a given ZNode to ZooKeeper.
Cluster-wide Configuration: Exhibitor attempts to present a single console for your entire ZooKeeper ensemble. Configuration changes made in Exhibitor will be applied to the entire ensemble.
Rolling Ensemble Changes: Exhibitor can update the servers in the ensemble in a rolling fashion so that the ZooKeeper ensemble can stay up and in quorum while the changes are being made.
Visualizer: Exhibitor provides a graphical tree view of the ZooKeeper ZNode hierarchy.
Curator Integration: Exhibitor and Curator (Cur/Ex!) can be configured to work together so that Curator instances are updated for changes in the ensemble.
Rich REST API: Exhibitor exposes a REST API for programmatic integration.
There are two versions of Exhibitor available to developers. You can either get the Standalone or Core version of the application. Standalone is a pre-configured Jetty-based self-contained application. Core is more flexible as it can be built into an existing application or one can build an extended application around it.
If you want to get into the nitty gritty of the code, Netflix has already posted the source code for Exhibitor on Github. Get cracking on some new applications using the Core version of Exhibitor.