The TensorFlow team has altered the basic way to do distributed training on TensorFlow. Now you can use TensorFlow 2.0, which turns on eager execution by default; refactor some of the core functionality out of Estimators; and pack up algorithms for distributing computation into slottable objects called DistributionStrategies.
Magnus Hyttsten explains how to use TensorFlow effectively in a distributed manner using best practices. Magnus covers using TensorFlow’s new DistributionStrategies to get easy high-performance training with Keras models (and custom models) on multi-GPU setups as well as multinode training on clusters with accelerators, explores some of the underlying algorithms (like Allreduce), and shows how this can accelerate your training in various hardware configurations. You’ll also learn how to measure performance and ways to consistently show and reproduce performance.
Magnus Hyttsten is a developer advocate for TensorFlow at Google, where he works on developing the TensorFlow product. A developer fanatic, Magnus is an appreciated speaker at major industry events such as Google I/O, the AI Summit, AI Conference, ODSC, GTC, QCon, and others on machine learning and mobile development. Right now, he’s focusing on reinforcement learning models and making model inference effective on mobile.
©2019, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org