Artificial neural networks (ANN) are popular models for machine learning, in particular for deep learning. The models that are used in practice for image classification and speech recognition contain a huge number of weights and are trained with big datasets, but training such models is challenging in terms of computation and data processing. Alexander Ulanov proposes a scalable implementation of deep neural networks for Spark.
Alexander addresses the computational challenge by employing batch operations, using BLAS for vector and matrix computations, and reusing the memory for reducing garbage collector activity. Spark provides data parallelism that enables scaling of training. As a result, the implementation is on par with widely used C++ implementations like Caffe on a single machine and scales nicely on a cluster. The developed API makes it easy to configure your own network and run experiments with different hyperparameters. Alexander’s implementation is easily extensible, and other developers are invited to contribute new types of neural network functions and layers.
Alexander covers optimizations applied and the experience with GPU CUDA BLAS, which might be useful for other machine-learning algorithms being developed for Spark. Additionally, Alexander provides estimates for the optimal number of nodes to be used for gradient descent training that correspond to experimental results.
Alexander Ulanov is a senior researcher at Hewlett Packard Labs, where he focuses his research on machine learning on a large scale. Currently, Alexander works on deep learning and graphical models. He has made several contributions to Apache Spark; in particular, he implemented the multilayer perceptron classifier. Previously, he worked on text mining, classification and recommender systems, and their real-world applications. Alexander holds a PhD in mathematical modeling from the Russian Academy of Sciences.
©2016, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.