Presented By O’Reilly and Cloudera
Make Data Work
September 11, 2018: Training & Tutorials
September 12–13, 2018: Keynotes & Sessions
New York, NY

Building a high-performance model serving engine from scratch using Kubernetes, GPUs, Docker, Istio, and TensorFlow

Chris Fregly (PipelineAI)
2:00pm–2:40pm Thursday, 09/13/2018
Data engineering and architecture, Expo Hall
Location: Expo Hall Level: Intermediate
Secondary topics:  Model lifecycle management
Average rating: ***..
(3.50, 2 ratings)

Who is this presentation for?

  • Data scientists, data engineers, data analysts, CTOs, and CEOs

Prerequisite knowledge

  • A basic understanding of machine learning

What you'll learn

  • Learn how to serve models in production the right way


Drawing on his experience at Netflix, particularly its culture, which encourages “freedom and responsibility,” Chris Fregly explains how data scientists can use PipelineAI to safely deploy their ML/AI pipelines into production using live data and details a full-featured, open source end-to-end TensorFlow model training and deployment system, using the latest advancements with Kubernetes, TensorFlow, and GPUs.

In addition to training and hyperparameter tuning, the model deployment pipeline includes continuous canary deployments of TensorFlow models into a live hybrid-cloud production environment. This is the holy grail of data science: rapid and safe experiments of ML/AI models directly in production.

Offline batch training and validation is for the slow and weak. Online real-time training and validation on live production data is for the fast and strong. Learn to be fast and strong by attending this talk.

Photo of Chris Fregly

Chris Fregly


Chris Fregly is an AWS Technical Evangelist for Machine Learning and AI based in San Francisco. He is founder of the Advanced KubeFlow Meetup and author of the O‚ÄôReilly Video Series titled, “High Performance TensorFlow in Production.” Previously, Chris was Founder and Product Manager at PipelineAI where he worked with many small startups and large enterprises to optimize and tune their ML/AI pipelines.