Edgestore is a low-latency, distributed data store that is one of the largest services developed at Dropbox. It serves 10 million requests per second and stores over 10 trillion objects at rest. In the last few years, Edgestore has grown from being used by a handful of services to being the primary data store for all of Dropbox’s metadata needs.
Mehant Baid discusses the challenges Dropbox faced as the company scaled Edgestore and its journey from being an operationally burdensome service that’s plagued with incidents and postmortems to a service that’s highly reliable and operationally lightweight.
Mehant begins with a brief introduction of Edgestore, then discusses the various ways Dropbox’s systems and processes failed as it scaled its service. He also shares the various technical solutions adopted at Dropbox, along with the trade-offs, to improve service reliability. Some of the problems he dives deep into include load management, client isolation, linearized reads from replicas, schema evolution, and data validation.
Mehant Baid is a software engineer at Dropbox. For the past few years, he’s been working on Edgestore, the distributed data store that handles all of Dropbox’s metadata needs. Previously, he worked on the database kernel at Oracle, where he focused on scaling inserts into the database. He’s a committer and project management committee member with the Apache Software Foundation and worked with the open source community to develop Apache Drill—an SQL engine for Hadoop, NoSQL, and cloud storage. His primary interests are the fields of distributed systems and databases.
©2019, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com