Presented By O’Reilly and Cloudera
Make Data Work
September 11, 2018: Training & Tutorials
September 12–13, 2018: Keynotes & Sessions
New York, NY

Building a high-performance model serving engine from scratch using Kubernetes, GPUs, Docker, Istio, and TensorFlow

Chris Fregly (PipelineAI)
2:00pm–2:40pm Thursday, 09/13/2018
Data engineering and architecture, Expo Hall
Location: Expo Hall Level: Intermediate
Secondary topics:  Model lifecycle management
Average rating: ***..
(3.50, 2 ratings)

Who is this presentation for?

  • Data scientists, data engineers, data analysts, CTOs, and CEOs

Prerequisite knowledge

  • A basic understanding of machine learning

What you'll learn

  • Learn how to serve models in production the right way


Drawing on his experience at Netflix, particularly its culture, which encourages “freedom and responsibility,” Chris Fregly explains how data scientists can use PipelineAI to safely deploy their ML/AI pipelines into production using live data and details a full-featured, open source end-to-end TensorFlow model training and deployment system, using the latest advancements with Kubernetes, TensorFlow, and GPUs.

In addition to training and hyperparameter tuning, the model deployment pipeline includes continuous canary deployments of TensorFlow models into a live hybrid-cloud production environment. This is the holy grail of data science: rapid and safe experiments of ML/AI models directly in production.

Offline batch training and validation is for the slow and weak. Online real-time training and validation on live production data is for the fast and strong. Learn to be fast and strong by attending this talk.

Photo of Chris Fregly

Chris Fregly


Chris Fregly is founder and research engineer at PipelineAI, a San Francisco-based streaming machine learning and artificial intelligence startup. Previously, Chris was a distributed systems engineer at Netflix, a data solutions engineer at Databricks, and a founding member of the IBM Spark Technology Center in San Francisco. Chris is a regular speaker at conferences and meetups throughout the world. He’s also an Apache Spark contributor, a Netflix Open Source committer, founder of the Global Advanced Spark and TensorFlow meetup, author of the upcoming book Advanced Spark, and creator of the O’Reilly video series Deploying and Scaling Distributed TensorFlow in Production.