Gimme More! Enabling User Growth in a Performant and Efficient Fashion

Operations and Culture
Location: Palace Suite Level: Intermediate
Average rating: ***..
(3.04, 23 ratings)

There has been an increasing impetus, in both academic and industry conferences such as Velocity, on the use of statistical and/or data mining techniques to achieve high performance and efficiency in data centers and the cloud. One of the challenges in this regard is to figure out which metrics to collect and how to mine the historical data of the same.

In the context of capacity planning – which plays a critical role in achieving high availability and performance – the metric space encompasses both system as well as application metrics. This, coupled with an ever changing product development landscape, makes capacity planning non-trivial. To this end, at Twitter, we have built a novel framework to facilitate visualization and analysis of a given set (in other words, the list of metrics is parameterized) of metrics. Specifically, the Analytics Engine of the framework employs a variety of time series analysis techniques (to, for example but not limited to, detect breakout in a time series) and forecasting techniques. Some of the techniques are enumerated below:

  • Momentum strategies
  • Beta analysis
  • Anomaly detection
  • ARIMA based forecasting

We shall also walk the audience through how we use the framework to determine the core underlying driver for a given resource/system constraint. We illustrate the use of the framework using REAL data.

Photo of Arun Kejariwal

Arun Kejariwal

Independent

@arun_kejariwal is currently a Staff Capacity Engineer at Twitter where he works on research and development of novel techniques to improve the accuracy of capacity models and demand forecasts. Prior to joining Twitter, @arun_kejariwal worked on research and development of practical and statistically rigorous methodologies to deliver high performance, availability and scalability in large scale distributed clusters. Some of the techniques developed have been published in peer-reviewed international conferences/journals.

@arun_kejariwal received his Bachelor’s degree in EE from IIT Delhi and doctorate in CS from UCI.

Winston Lee

Twitter Inc.

@winstl is a Capacity Engineer at Twitter. He recently graduated from UC, Berkeley with CS major. Amongst others, he is highly interested in cloud computing and BigData analysis.

Sponsors

Sponsorship Opportunities

For exhibition and sponsorship opportunities at Velocity conference, contact Gloria Lombardo at +1 (203) 381-9245 or glombardo@oreilly.com

Media Partner Opportunities

For information on trade opportunities with O'Reilly conferences contact Jaimey Walking Bear at mediapartners
@oreilly.com

Contact Us

View a complete list of Velocity Europe 2013 contacts