Put AI to Work
April 15-18, 2019
New York, NY

The curse of generality: Deep reinforcement learning in the wild

Sanjay Krishnan (University of Chicago)
2:40pm3:20pm Wednesday, April 17, 2019
Machine Learning, Models and Methods
Location: Grand Ballroom West
Secondary topics:  Automation in machine learning and AI, Models and Methods, Reinforcement Learning, Reliability and Safety
Average rating: ***..
(3.00, 2 ratings)

What you'll learn

  • Explore the underappreciated challenges to implementing deep reinforcement learning


The ubiquity of sequential decision problems throughout computer science makes deep reinforcement learning one of the most exciting developments of modern AI. However, realizing the potential of such general frameworks in real applications has proven to be much more challenging.

Drawing on his work building and deploying an RL-based relational query optimizer, a core component of almost every database system, Sanjay Krishnan highlights some of the underappreciated challenges to implementing deep reinforcement learning. RL algorithms today do not fully exploit the structure of software simulators by collecting data episodically instead of strategically rewinding, fast-forwarding, and skipping. Further, they are very sensitive to policy parametrization especially in cases where there are hierarchical or discontinuous policy structures. RL algorithms also struggle in “overactuated” problems where the action space has significant redundancy. For all three of these challenges, Sanjay shares experimental results illustrating phenomena in practice, along with algorithmic solutions and overviews of the ways the same phenomena appear in other RL domains, such as robotics.

Photo of Sanjay Krishnan

Sanjay Krishnan

University of Chicago

Sanjay Krishnan is an assistant professor of computer science at the University of Chicago. His research focuses on applications of machine learning and control theory to computer and cyberphysical systems problems. His work has received a number of awards including the 2016 SIGMOD Best Demonstration award, 2015 IEEE GHTC Best Paper award, and Sage Scholar award. Sanjay holds a PhD and master’s degree in computer science from UC Berkeley.