TensorFlow and TPUs in the real world: Converting deep learning projects to train faster

Sam Witteveen (Red Dragon AI)

11:00am–11:40am Wednesday, October 30, 2019

Location: Grand Ballroom H

Accelerators

Who is this presentation for?

Developers and deep learning engineers

Level

Intermediate

Description

TPUs are the quickest way to train large deep learning models and, as of TensorFlow 2.0, they’ve become much easier to use with tf.keras and distribution strategies. Sam Witteveen explains how to take a deep learning project and convert it to run on cloud TPUs so it trains faster and cheaper than using GPUs in your own box or in the cloud.

Sam dives into a variety of best practices for writing code optimized for TPUs and distribution strategies, including using Google Cloud to run TensorFlow, training with TPUs on the Cloud ML Engine, using tf.data to build data pipelines that are TPU friendly, customizing your model to make it optimized for TPUs, using TensorFlow distribution strategies, using Colab to test out your model for free, the difference between different versions of TPUs, and using the TPU profiler to monitor your training and acceleration.

Prerequisite knowledge

A basic understanding of TensorFlow

What you'll learn

Discover how to use TPUs easily to train a variety of models, how to use TPUs on Cloud ML Engine, and the difference between different versions of TPUs
Learn how to use tf.data code to build data pipelines, customize your model code, use Colab to test out your model for free, and use the TPU profiler to monitor your training and acceleration

Sam Witteveen

Red Dragon AI

Sam Witteveen is a developer expert for machine learning at Google. He has extensive experience in startups and mobile applications and helps developers and companies create smarter applications with machine learning. He’s especially passionate about deep learning and AI in the fields of natural language and conversational agents. Sam regularly shares his knowledge at events and trainings across Asia and is co-organizer of the Singapore TensorFlow and Deep Learning group.