14–17 Oct 2019

Using reinforcement learning to build recommendation systems with AWS SageMaker RL

Sergey Ermolin (Amazon Web Services), Vineet Khare (Amazon Web Services)
13:3017:00 Tuesday, 15 October 2019
Location: Windsor Suite
Average rating: *....
(1.50, 4 ratings)



Using reinforcement learning for recommendation systems is a fairly novel field of academic studies. Sergey Ermolin and Vineet Khare provide a step-by-step overview on how to implement, train, and deploy a reinforcement learning (RL)-based recommender system with real-time multivariate optimization. They show you how leverage RL to implement a recommender system that optimizes an advertisement message that promotes adoption of merchant’s services.


  • Introduction to reinforcement learning and recommender system concepts
  • AWS account setup and SageMaker RL environment configuration
  • Recommender system use case (based on this KDD-2017 paper): problem statement, algorithm outline, implementation outline, simulation environment outline, and results and references
  • Detailed walk-through of the prepared Python Jupyter notebook with algorithm implementation
  • Step-by-step Jupyter notebook execution
  • Results of simulation experiments
  • Analysis of the results and suggestions for further experiments
  • Discussion, conclusions, closing remarks, and Q&A

Prerequisite knowledge

  • Familiarity with Python and Jupyter notebooks
  • A basic understanding of reinforcement learning concepts (useful but not required)

What you'll learn

  • Learn to apply reinforcement learning to RecSys use cases and use an end-to-end reinforcement learning platform to rapidly prototype and implement a typical use case

Sergey Ermolin

Amazon Web Services

Sergey Ermolin is a principal solutions architect (ML/DL/AI) for Amazon Web Services. Previously, he was a software solutions architect for deep learning, Spark analytics, and big data technologies at Intel. A Silicon Valley veteran with a passion for machine learning and artificial intelligence, Sergey has been interested in neural networks since 1996, when he used them to predict aging behavior of quartz crystals and cesium atomic clocks made by Hewlett-Packard. Sergey holds an MSEE and a certificate in mining massive datasets from Stanford and BS degrees in both physics and mechanical engineering from California State University, Sacramento.

Photo of Vineet Khare

Vineet Khare

Amazon Web Services

Vineet Khare is a manager of applied science at AWS, where he’s led the research and development efforts for multiple AWS products, including SageMaker built-in algorithms, SageMaker RL, AWS DeepRacer, and AWS Ground Truth. He’s presented his research at international conferences including PPSN, EMO, GECCO, and SEAL. He’s also conducted SageMaker workshops and tutorials at AWS events such as the annual conference, re:Invent.

  • Intel AI
  • O'Reilly
  • Amazon Web Services
  • IBM Watson
  • Dell Technologies
  • Hewlett Packard Enterprise
  • AXA

Contact us


For conference registration information and customer service


For more information on community discounts and trade opportunities with O’Reilly conferences


For information on exhibiting or sponsoring a conference


For media/analyst press inquires