Skip to main content

Exascale Data Analytics @ Facebook

Data in Action
Ballroom CD
Average rating: ****.
(4.36, 11 ratings)

Data analytics is at the heart of product development at Facebook. Roughly 1000 people across the company – technical and non-technical – use the Analytics Infrastructure every day to derive insights and metrics on products, ads and for other business data analysis. Facebook’s data warehouse has grown rapidly over the years, and poses unique scalability challenges (the warehouse is currently more than 250 PB in size, grew 3x in the last year and crunches more than 10 PB of data a day). The Analytics Infrastructure team at Facebook has been continuously evolving the system to handle the growing needs, both on storage and compute.

This talk will briefly outline the evolution of the analytics software stack in the last year and delve deeper into the data management and compute challenges and solutions at this scale. Specifically, the talk will focus on the evolution of Hive for storage and compute efficiency; and discuss the key ideas, observations and experiences in applying it to the Facebook data warehouse. In-depth topics that will be covered include storage format evolution for enhanced data compression, increasing Hadoop cluster utilization and managing multi-tenancy in the clusters.

Sambavi Muthukrishnan

Engineering Manager, Facebook

Sambavi Muthukrishnan is an Engineering Manager in the Analytics Infrastructure group at Facebook. She leads the development of batch analytic engines for one of the largest data warehouses in the world. Her team works on the core of Hive and Hadoop (Corona) – evolving the query engine, compute framework and storage to operate at scale; as well as on Giraph (graph analytics). Previously, she was an Engineering Manager at Microsoft in the SQL Server group where she worked on query processing.

Recent talks by Sambavi:

  • Analytics @ WebScale conference at Facebook on June 2013
  • Panelist on ‘Management of Big Data Systems’ panel at Usenix ICAC July 2013
  • Industry vision talk at VLDB 2013 (Aug 2013)