Presented By O'Reilly and Cloudera
Make Data Work
31 May–1 June 2016: Training
1 June–3 June 2016: Conference
London, UK

Beautiful A/B testing

Marton Trencseni (Facebook)
14:05–14:45 Friday, 3/06/2016
Data-driven business
Location: Capital Suite 4 Level: Non-technical
Average rating: ***..
(3.12, 8 ratings)

At first glance A/B testing is a simple matter: take a few numbers, put them into an online calculator, and read off the statistical significance. But in fact it’s a complex topic with amazing opportunities (and pitfalls) for organizations. Marton Trencseni offers a deep dive into A/B testing to provide attendees, covering lessons learned and offering suggestions how to avoid and address challenges, with the goal of raising awareness and providing attendees the information needed to improve their organizations’ experimentation cultures.

Topics include:

  • Frequentist vs. Bayesian A/B testing
  • Random vs. nonrandom bucketing
  • The peeking pitfall
  • Data mining segments and KPIs
  • Daily variation and seasonality
  • Raw lift vs. projected lift
Photo of Marton Trencseni

Marton Trencseni


Marton Trencseni is a data engineer at Facebook in London. An engineer and physicist by training, Marton worked in various engineering roles before launching his own distributed database startup. Then Marton joined Prezi and spent three years as director of data analytics, building out Prezi’s data platform and data team. Marton regularly shares stories and practical knowledge with the data community at conferences.