Presented By
O’Reilly + Intel AI
Put AI to Work
April 15-18, 2019
New York, NY

“The Curse of Generality”: Deep Reinforcement Learning in the Wild

Sanjay Krishnan (University of Chicago)
2:40pm3:20pm Wednesday, April 17, 2019
Machine Learning, Models and Methods
Location: Grand Ballroom West
Secondary topics:  Automation in machine learning and AI, Models and Methods, Reinforcement Learning, Reliability and Safety

The ubiquity of sequential decision problems throughout Computer Science makes Deep Reinforcement Learning one of the most exciting developments of modern AI. However, realizing the potential of such general frameworks in real applications has proven to be much more challenging. I use my work over the last few years on building and deploying an RL-based relational query optimizer, a core component of almost every database system, as an exemplary application that highlights some of the under-appreciated challenges in Deep RL practice. RL algorithms today: (1) do not fully exploit the structure of software simulators by collecting data episodically instead of strategically rewinding, fast-forwarding, skipping, (2) are very sensitive to policy parametrization especially in cases where there are hierarchical or discontinuous policy structures, and (3) struggle in “over-actuated” problems where the action space has significant redundancy. For all three challenges, I present experimental results illustrating phenomena in practice, our algorithmic solutions, and highlight the ways in which the same phenomena appear in other RL domains such as robotics.

Photo of Sanjay Krishnan

Sanjay Krishnan

University of Chicago

Sanjay Krishnan is an Assistant Professor of Computer Science at the University of Chicago. His research focuses on applications of machine learning and control theory to computer and cyber-physical systems problems. Sanjay completed his PhD and Masters Degree at UC Berkeley in Computer Science in 2018. Sanjay’s work has received a number of awards including the 2016 SIGMOD Best Demonstration award, 2015 IEEE GHTC Best Paper award, and Sage Scholar award.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)