Presented By O’Reilly and Intel AI
Put AI to Work
April 29-30, 2018: Training
April 30-May 2, 2018: Tutorials & Conference
New York, NY

Scalable deep learning

Ameet Talwalkar (Carnegie Mellon University | Determined AI)
2:35pm–3:15pm Tuesday, May 1, 2018
Models and Methods
Location: Grand Ballroom East
Average rating: ****.
(4.50, 2 ratings)

Who is this presentation for?

  • Machine learning engineers, data scientists, and data platform engineers

Prerequisite knowledge

  • A basic understanding of deep learning

What you'll learn

  • Explore approaches for addressing two core scalability bottlenecks: tuning the knobs of deep learning models (i.e., hyperparameter optimization) and training deep models in parallel environments


Although deep learning has received much acclaim due to its widespread empirical success, fundamental bottlenecks exist when attempting to develop deep learning applications at scale. One involves exploring the design space of a model family, which typically requires training tens to thousands of models with different hyperparameters. Model training itself is a second major bottleneck, as classical learning algorithms are often infeasible for the petabyte datasets that are fast becoming the norm.

Ameet Talwalkar shares research on addressing these two core scalability bottlenecks. Ameet first offers an overview of Hyperband, a novel algorithm for hyperparameter optimization that is simple, flexible, theoretically sound, and an order of magnitude faster than leading competitors. He then presents work aimed at understanding the underlying landscape of training deep learning models in parallel and distributed environments and introduces Paleo, an analytical performance model that can quickly and accurately model the expected scalability and performance of putative parallel and distributed deep learning systems.

Photo of Ameet Talwalkar

Ameet Talwalkar

Carnegie Mellon University | Determined AI

Ameet Talwalkar is cofounder and chief scientist at Determined AI and an assistant professor in the School of Computer Science at Carnegie Mellon University. His research addresses scalability and ease-of-use issues in the field of statistical machine learning, with applications in computational genomics. Ameet led the initial development of the MLlib project in Apache Spark. He is the coauthor of the graduate-level textbook Foundations of Machine Learning (MIT Press) and teaches an award-winning MOOC on edX, Distributed Machine Learning with Apache Spark.