Build Systems that Drive Business
Sep 30–Oct 1, 2018: Training
Oct 1–3, 2018: Tutorials & Conference
New York, NY

Smooth scaling: Slack’s journey toward a new database

Ameet Kotian (Slack)
11:35am–12:15pm Tuesday, October 2, 2018
Distributed Data
 Location: Nassau Level: Intermediate
Secondary topics:  Resilient, Performant & Secure Distributed Systems
Average rating: *****
(5.00, 1 rating)
Download slides (ZIP)

Prerequisite knowledge

  • A basic understanding of distributed systems and distributed databases

What you'll learn

  • Learn how Slack uses Vitess, a powerful open source database cluster solution for MySQL

Description

Slack has experienced tremendous growth for a young company, serving over nine million weekly active customers. But with great growth comes greater growth pains. Slack’s rapid growth over the last few years outpaced the scaling capacity of its original sharded MySQL database, which negatively impacted the company’s customers and engineers.

Ameet Kotian explains how a small team of engineers embarked on a journey for the right database solution, which eventually led them to Vitess, a powerful open source database cluster solution for MySQL. Vitess combines the features of MySQL with the scalability of a NoSQL database. It has been serving Youtube’s traffic for numerous years and has a strong community.

Although Vitess meets a lot of Slack’s needs, it’s not an out-of-the-box solution. Ameet shares how the journey to Vitess was planned and executed, with little customer impact, in the face of piling operational challenges, such as AWS issues, MySQL replication, automatic failovers, deployments strategies, and so forth. Ameet also covers Vitess’s architecture, trade-offs, and what the future of Vitess looks like at Slack.

Photo of Ameet Kotian

Ameet Kotian

Slack

Ameet Kotian is a senior storage operations engineer at Slack, where he is responsible for operating the infrastructure used to store all of Slack’s data. Previously, he was one of the first site reliability engineers at Twitter, where he spent close to five years working on the graph storage service and Twitter’s internal distributed database, Manhattan. His work related to large-scale automated deployments was featured on Twitter’s engineering blog.