Depending on how they are communicated, data-driven business decisions can be the secret to a business’ success or the reason for its failure. The excitement over big data and the numerous analytical and machine-learning techniques to analyze and predict outcomes can divert decision makers from the fundamental fact that most big data is still random and possibly biased. There has never been a more important time to communicate uncertainty in data effectively.
Abigail Lebrecht discusses the concept of uncertainty within the context of probability theory and highlights some of the common pitfalls that can lead to “dodgy” data and misinformed business decisions, through recent examples from science, the media, and business. Abigail then offers an introduction to Bayes’ theorem, which provides the foundations of many of the machine-learning methods employed today, and explains the importance of considering conditionality when thinking about your data and its structure, sharing examples of bad decisions when conditionality is not considered.
Abigail outlines both Bayesian and frequentist approaches to understanding uncertainty, specifically looking at their different applications in the context of A/B tests (one of the most common areas in which communicating uncertainty is fundamental). Abigail explains why many statisticians have strong opinions on each and compares frequentist and Bayesian approaches for ensuring confidence in your results.
Abigail Lebrecht is principal analyst at uSwitch, where she focuses on using statistical and machine-learning techniques for descriptive analytics and modeling to understand customer behavior. Abigail has a background in probability and statistics and is passionate about encouraging an understanding of uncertainty in both big and small data. She holds a PhD in queueing theory from Imperial College London.
©2016, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.