October 28–31, 2019

TensorFlow on the Cerebras Wafer-Scale Engine

Manjunath Kudlur (Cerebras Systems), Andy Hock (Cerebras Systems)
5:00pm5:40pm Wednesday, October 30, 2019
Location: Grand Ballroom H

Who is this presentation for?

  • Machine and deep learning researchers and application developers, software engineers, TensorFlow and other framework engineers, computer hardware developers, and product and business leaders

Level

Intermediate

Description

The Cerebras WSE is the largest chip ever built and the central processor of a new computer system purpose-built to accelerate deep learning. The WSE delivers more compute, more memory, and more communication bandwidth to enable AI research at revolutionary speeds and scale.

Manjunath Kudlur and Andy Hock describe the software stack that connects users and TensorFlow to the Cerebras WSE. The Cerebras stack takes standard code from TensorFlow as input and automatically generates an optimized executable for the WSE. This gives you access to cluster-scale performance in a single node without changes to programming paradigm.

They provide an overview of the compilation process, including major components and the state of tools for the TF development community. They also describe the approach to extract parallelism from large graphs to a large chip: how the compiler makes decisions, for example, about mapping user-defined model layers to compute and memory and layer connectivity to communication across the chip.

Prerequisite knowledge

  • A working knowledge of TensorFlow and how to build and train models
  • Knowledge of XLA, estimators within TensorFlow, and how to run training on other accelerators such as GPU or TPU (useful but not required)

What you'll learn

  • Take a first look at Cerebras's software for its recently announced accelerator
  • Gain insight to software architecture and neural network compilation for a new chip at a truly unique scale
  • Hear ideas for training acceleration possibilities beyond what's achievable in existing hardware and ideas for model architectures beyond what's possible to accelerate in existing hardware
Photo of Manjunath Kudlur

Manjunath Kudlur

Cerebras Systems

Manjunath Kudlur is the technical lead for Cerebras Systems’s compiler software project, mapping neural networks to a revolutionary new deep learning accelerator with a wafer-scale processor. He’s an engineer with expertise in compilers, machine learning, and parallel computing. Previously, he worked at Google in Brain on TensorFlow and at NVIDIA on compilers and programming languages research. Manjunath has a PhD in computer science and engineering from the University of Michigan.

Photo of Andy Hock

Andy Hock

Cerebras Systems

Andy Hock is the director of product for Cerebras Systems, an AI hardware startup out to accelerate deep learning and change compute forever. He has 10 years of experience in product management, technical program management, and enterprise business development; over 15 years of experience in research, algorithm development, and data analysis for image processing; and 5 years’ experience in applied machine learning and AI. Previously, Andy was the product manager of data and analytics for Terra Bella at Google, where he led the development of machine learning-powered data products from satellite imagery; was senior director for advanced technology programs at Skybox Imaging (which became Terra Bella following acquisition by Google in 2014); and was a senior program manager and senior scientist at Areté. He has a PhD in geophysics and space physics from the University of California, Los Angeles, and a BA in astronomy-physics from Colgate University.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

sponsorships@oreilly.com

For information on exhibiting or sponsoring a conference

pr@oreilly.com

For media/analyst press inquires