Today’s open source databases are plentiful and offer wildly different capabilities. As technology companies push the boundaries of what traditional RDBMS can do to the limit, we’ve seen significant innovation in open source “distributed first” data stores, including key value stores, search engines, document stores, caches, and even distributed locking systems. Joseph Lynch explores how Yelp made the hard technical choices and built a bulletproof data tier from these distributed data stores.
Joseph starts with a survey of the open source datastore landscape, outlining the high-level trade-offs that have to be made when choosing between different classes (e.g., relational versus NoSQL) of data stores, as well as the limitations of those choices. Joseph then explains how Yelp made the decision for search engines (Elasticsearch versus Solr); configuration systems (Zookeeper versus Etcd); key value store (Cassandra versus HBase); and caching (MySQL versus Cassandra versus Memcache versus Redis).
Regardless of which set of open source data stores a company chooses, the hard part is getting it to production. In order to keep up with all the new options, Yelp invested early in building a common platform for deploying, configuring, and monitoring data stores. Joseph discusses some of these shared abstractions including:
Joseph ends by covering the implications of giving developers so many choices in your data store infrastructure and sharing some lessons learned about the requisite education in a DevOps datastore world. Companies are scaling and iterating far beyond the days where one could run a single database cluster, and just as monoliths are becoming microservices, catch-all databases are turning into polyglot data stores.
Joseph Lynch is a software engineer for Yelp who focuses on building data store and service infrastructure. Joey is a core contributor to Yelp’s data store platform, which has allowed Yelp to go from a primarily MySQL data tier to a polyglot data tier including Elasticsearch, Cassandra, and Zookeeper. He loves pushing the edge of how Yelp uses DevOps tools to automate infrastructure and never met a problem he didn’t want to automate away. When not wrangling clusters of data stores, Joey enjoys building service discovery, reliable communication, fast deployment, and monitoring into Yelp’s SOA.
©2016, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com