Presented By
O’Reilly + Cloudera
Make Data Work
March 25-28, 2019
San Francisco, CA
Please log in

How to make fewer bad decisions

Eric Colson (Stitch Fix), Daragh Sibley (Stitch Fix)
2:40pm3:20pm Wednesday, March 27, 2019
Average rating: ****.
(4.79, 14 ratings)

Who is this presentation for?

  • Data scientists, executives, decision makers, and domain experts



Prerequisite knowledge

  • Familiarity with business decision making, A/B testing, and empirical methods

What you'll learn

  • Understand the value of A/B testing and domain expertise
  • Learn the types of cognitive biases to be aware of and mechanisms to mitigate your intuitions


The increasing prevalence of A/B testing has revealed a powerful but seldom discussed insight: Our intuitions are soberingly bad. Yet most of us are either unaware or choose not to believe this. Time and time again, randomized controlled trials reveal that our predictions on the outcomes of business decisions are deeply fallible. Experiments show that our optimism in new features is often misguided, what we thought would help actually hurts, and what we believed to be the next big thing fails completely. Our intuitions can even lead to worse outcomes than what we would expect from random chance alone.

This is not a talk about A/B testing but rather about what empiricism reveals about our intuitions. Eric Colson and Daragh Sibley describe some types of errors domain experts commit, explain how these systematic errors arise from heuristic reasoning processes that served us well on an evolutionary time scale but now impair our judgments, and illustrate how cognitive heuristics can compromise decisions and their evaluation.

While we can’t change how our brains are wired, knowledge of how they work can give us an advantage. Eric and Daragh share several mechanisms to mitigate these human limitations and improve our decision making and argue that businesses that make fewer bad decisions will have a distinct competitive advantage.

Photo of Eric Colson

Eric Colson

Stitch Fix

Eric Colson is chief algorithms officer at Stitch Fix, where he leads a team of 100+ data scientists and is responsible for the multitude of algorithms that are pervasive to nearly every function of the company, from merchandise, inventory, and marketing to forecasting and demand, operations, and the styling recommender system. He’s also an advisor to several big data startups. Previously, Eric was vice president of data science and engineering at Netflix. He holds a BA in economics from SFSU, an MS in information systems from GGU, and an MS in management science and engineering from Stanford.

Photo of Daragh Sibley

Daragh Sibley

Stitch Fix

Daragh Sibley is director of data science at Stitch Fix, where he leads a team of data scientists that use algorithms and the scientific method to optimize the portfolio of products stocked in Stitch Fix’s inventory. Previously, Daragh spent a decade in academia, where he developed neural networks of human language acquisition and tested their predictions with behavioral and neuroimaging experiments.

Comments on this page are now closed.


Shanthi Ravi
03/27/2019 8:36am PDT

Was this session recorded? or can we get the slides?