October 28–31, 2019
Please log in

Running TFX end to end in hybrid clouds leveraging Kubeflow Pipelines

Animesh Singh (IBM), Pete MacKinnon (Red Hat), Tommy Li (IBM)
11:00am11:40am Wednesday, October 30, 2019
Location: Grand Ballroom C/D

Who is this presentation for?

  • Senior technical staff managers and program directors




TFX is an end-to-end platform for deploying production ML pipelines. It provides a configuration framework and shared libraries to integrate common components needed to define, launch, and monitor your machine learning system. Kubeflow Pipelines enable composition and execution of reproducible workflows on Kubeflow, integrated with experimentation and notebook-based experiences.

Animesh Singh, Pete MacKinnon, and Tommy Li demonstrate how to run TFX in hybrid cloud environments with best practices while leveraging Kubeflow Pipelines to provide a single atomix unit spanning multiple clouds. Given that TFX traverses multiple phases of AI lifecycle including data validation, data transformation, model analysis, training, serving, etc., these parts of the TFX lifecycle are often carried out in hybrid cloud environments. For example, data validation and data transformation happen on-premises, whereas model training and serving happen in a public cloud environment.

Prerequisite knowledge

  • Familiarity with TFX and AI pipelines

What you'll learn

  • Understand TFX, hybrid cloud ML pipelines, and Kubeflow
Photo of Animesh Singh

Animesh Singh


Animesh Singh is a senior technical staff manager and program director at IBM, leading IBM AI OSS strategy working with the IBM Watson and Cloud Platform. He leads machine learning and deep learning initiatives and works with communities and customers to design and implement deep learning, machine learning, and cloud computing frameworks. He has a proven track record of driving design and implementation of private and public cloud solutions from concept to production. In his decade-plus at IBM, Animesh has worked on cutting-edge projects for IBM enterprise customers in the telco, banking, and healthcare industries, particularly focusing on cloud and virtualization technologies, and led the design and development of the first IBM public cloud offering.

Photo of Pete MacKinnon

Pete MacKinnon

Red Hat

Pete MacKinnon is a principal software engineer in the AI Center of Excellence at Red Hat. He’s actively involved in the open source Kubeflow project to bring TensorFlow machine learning workloads to container environments (Kubernetes and OpenShift).

Photo of Tommy Li

Tommy Li


Tommy Li is a software developer at IBM focusing on cloud, container, and infrastructure technology. He’s worked on various developer journeys that provide use cases on cloud-computing solutions, such as Kubernetes, microservices, and hybrid cloud deployments. He’s passionate about machine learning and big data.

  • O'Reilly
  • TensorFlow
  • Google Cloud
  • IBM
  • Databricks
  • Tensor Networks
  • VMware
  • Amazon Web Services
  • One Convergence
  • Quantiphi
  • Lambda Labs
  • Tech Mahindra
  • cnvrg.io
  • Determined AI
  • Inferencery
  • Manceps, Inc.
  • PerceptiLabs
  • Valohai

Contact us


For conference registration information and customer service


For more information on community discounts and trade opportunities with O’Reilly conferences


For information on exhibiting or sponsoring a conference


For media/analyst press inquires