Over the past decade, the industry has invested heavily in hardware accelerators like GPU and TPU. While these investments are essential in solving the hard machine learning problems, they are not sufficient to close the gap created by increasing data sizes and death of Moore’s law. Two legends of Computer Science, John Hennessy, and Dave Paterson, in their Turing lecture claim that it is indeed possible to speed up computations purely by software optimizations to the order of 61k times. However, in practice is a hard task. Even a 10x improvement would be a great win. RocketML has done exactly that.
Vinay Rao, CEO and Santi Adavani, CTO of RocketML explains why software architectures will lead next generation of machine learning approaches. The speaker takes you through the history of both distributed machine learning and hardware architectures, details the breakthroughs that have been made in software, and demonstrates how software ONLY approaches can ultimately scale better.
In this talk, we present a parallelized implementation of the L-BFGS algorithm on a distributed system which includes a cluster of commodity computing machines. We use open source HPCC Systems (High-Performance Computing Cluster) platform as the underlying distributed system to implement the L-BFGS algorithm. We initially provide an overview of the HPCC Systems framework and how it allows for the parallel and distributed computations important for Big Data analytics and, subsequently, we explain our implementation of the L-BFGS algorithm on this platform. Our experimental results show that our large-scale implementation of the L-BFGS algorithm can easily scale from training models with millions of parameters to models with billions of parameters by simply increasing the number of commodity computational nodes.
Founder and CEO of RocketML, a machine learning platform. We are on a mission to lead and enable transformation of the world towards Artificial Intelligence (AI)
Santi co-founded RocketML, where his team is building a superfast engine for building machine learning models. Before that, Santi worked as a Product manager and software development lead at Intel’s technology and manufacturing group. Prior to Intel, he got his Ph.D. in computational sciences from the University of Pennsylvania. His areas of expertise include high-performance computing, non-linear optimization, partial differential equations, machine learning, and big data.
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
©2019, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com