Presented By
O’Reilly + Cloudera
Make Data Work
March 25-28, 2019
San Francisco, CA
Please log in

Efficient multi-armed bandit with Thompson sampling for applications with delayed feedback

Shradha Agrawal (Adobe)
4:40pm5:20pm Thursday, March 28, 2019
Secondary topics:  Media, Marketing, Advertising
Average rating: ****.
(4.17, 6 ratings)

Who is this presentation for?

  • Data scientists, analysts, and engineers



Prerequisite knowledge

  • A basic understanding of data science and engineering

What you'll learn

  • Learn the basics of reinforcement learning
  • Explore an application using real-time data


The exploration-exploitation trade-off is a fundamental dilemma in online decision making. Reinforcement learning (RL) approaches are often employed to achieve optimal outcomes. Multi-armed bandits (MAB) are popular RL algorithms tailored for tackling the exploration-exploitation trade-off. However, increasing the number of arms (i.e., decision criteria) leads to exponential increase in complexity. Multi-armed bandits need a fast feedback loop to be able to improve their policy decisions and converge to the optimal solution, but delayed feedback is common in many applications—for example, in advertising, information about conversion would be available long after the advertisement was displayed.

Shradha Agrawal offers an overview of MABs and explains how to efficiently scale to multiple decision criteria. Shradha focuses on the Thompson sampling technique, which uses randomization effectively to handle observational delays—using an example from advertising to show how the solution can be used to provide relevant and personalized experiences to users in real-time to increase conversions.

Photo of Shradha Agrawal

Shradha Agrawal


Shradha Agrawal is a data scientist at Adobe in San Jose. She holds a master’s degree in computer science with a focus on AI and machine learning from the University of California, San Diego. She is the author of a number of papers and patent applications.

Comments on this page are now closed.


Picture of Shradha Agrawal
Shradha Agrawal | DATA SCIENTIST
04/02/2019 10:23am PDT

Please download the slides from this link-

03/29/2019 4:33am PDT

Fantastic talk! Would you be able to share the slides?