GPUs have proven their value for machine learning, offering orders-of-magnitude speedups on dense and sparse data. They define the current performance limits for machine learning but have limited model capacity. John Canny explains how to mitigate that challenge and achieve linear speedups with GPUs on commodity networks. The result defines the hitherto unseen “outer limits” of ML performance.
BIDMach is a rooflined machine-learning toolkit that has demonstrated a two-orders-of-magnitude gain over other systems when run on GPU hardware. BIDMach often outperforms cluster systems on problems where models fit in GPU memory and defines the performance limit for small-model problems. But there is great demand in industry for larger models, and BIDMach has migrated to the Spark ecosystem to boost model capacity. Scaling BIDMach is not an easy task, however. With two-orders-of-magnitude more computation at each node, a proportionate increase in network bandwidth is needed to avoid stifling performance. Or we need to find radical ways to reduce the load on the network.
John describes two techniques that each give us a roughly order-of-magnitude reduction in network load for common ML algorithms on power-law data. Together they allow us to mitigate network load and exploit GPU performance fully for both data-parallel and model-parallel calculations on commodity clusters. John builds on his earlier work on layered, heterogeneous hypercube networks (Kylix) to add topology-aware optimization and an extremely efficient error-tolerance protocol. BIDMach’s single node performance has also increased dramatically over the last year through the design of end-to-end learning algorithms. These algorithms—word2vec is one example—exploit GPU’s massive register storage (several megabytes) to provide dense computation speeds (600 Gflops) on sparse data problems.
John summarizes some other developments on BIDMach including:
John F. Canny is a computer scientist and the Paul and Stacy Jacobs Distinguished Professor of Engineering in the Computer Science Department of the University of California, Berkeley. John has made significant contributions in various areas of computer science and mathematics, including artificial intelligence, robotics, computer graphics, human-computer interaction, computer security, computational algebra, and computational geometry.
©2016, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.