Build & maintain complex distributed systems
October 1–2, 2017: Training
October 2–4, 2017: Tutorials & Conference
New York, NY

How Twitter built a framework to improve infrastructure utilization and efficiency at scale

Vinu Charanya (Twitter)
4:45pm–5:25pm Wednesday, October 4, 2017
Capacity Planning, Real time, events, streams & scale
 Location: Grand Ballroom West
Average rating: ****.
(4.00, 1 rating)

Who is this presentation for?

  • Engineering managers (platform and infrastructure), site reliability engineers, capacity planners, CIOs, and members of finance teams

What you'll learn

  • Explore Twitter's system for capturing, defining, provisioning, metering, and charging infrastructure resources
  • Discover how data from this system has improved overall resource utilization and led to significant savings in infrastructure spend

Description

Twitter is powered by thousands of microservices running on an internal cloud platform consisting of a suite of multitenant platform services that offer compute, storage, messaging, monitoring, etc. as a service. These platforms have thousands of tenants and run atop hundreds of thousands of servers both on-premises and in the public cloud. The scale of diversity in Twitter’s multitenant infrastructure services makes it extremely difficult to effectively forecast capacity, compute resource utilization, and cost and drive efficiency.

Vinu Charanya explains how she and her team are building a system that captures, defines, provisions, meters, and charges infrastructure resources, redefining how systems are built atop Twitter infrastructure. The infrastructure resources include primitive bare metal servers and VMs in the public cloud and abstract resources offered by multitenant services such as a compute platform (powered by Apache Aurora and Mesos), storage (Manhattan for key-value, cache, RDBMS), and observability. Along the way, Vinu shares how Twitter used this data to better plan capacity and drive a cultural change in engineering that helped improve overall resource utilization and led to significant savings in infrastructure spend.

Photo of Vinu Charanya

Vinu Charanya

Twitter

Vinu Charanya is a senior software engineer at Twitter, where she is building Twitter’s internal cloud infrastructure management platform. Vinu is also a core team member of Women Who Code, a nonprofit organization dedicated to inspiring women to excel in technology careers. She is also a part-time teacher of iOS and Android development and a mentor at CodePath. She holds a master’s degree in computer science and engineering from the University at Buffalo, where she worked in the PhoneLab Testbed research group under Geoff Challen and Steve Ko.