Presented By O'Reilly and Cloudera
Make Data Work
March 28–29, 2016: Training
March 29–31, 2016: Conference
San Jose, CA

A scalable implementation of deep learning on Spark

Alexander Ulanov (Hewlett Packard Labs)
11:00am–11:30am Tuesday, 03/29/2016
Hardcore Data Science
Location: 210 C/G
Average rating: ***..
(3.64, 11 ratings)

Prerequisite knowledge

Attendees should have a basic understanding of Apache Spark and machine learning.

Description

Artificial neural networks (ANN) are popular models for machine learning, in particular for deep learning. The models that are used in practice for image classification and speech recognition contain a huge number of weights and are trained with big datasets, but training such models is challenging in terms of computation and data processing. Alexander Ulanov proposes a scalable implementation of deep neural networks for Spark.

Alexander addresses the computational challenge by employing batch operations, using BLAS for vector and matrix computations, and reusing the memory for reducing garbage collector activity. Spark provides data parallelism that enables scaling of training. As a result, the implementation is on par with widely used C++ implementations like Caffe on a single machine and scales nicely on a cluster. The developed API makes it easy to configure your own network and run experiments with different hyperparameters. Alexander’s implementation is easily extensible, and other developers are invited to contribute new types of neural network functions and layers.

Alexander covers optimizations applied and the experience with GPU CUDA BLAS, which might be useful for other machine-learning algorithms being developed for Spark. Additionally, Alexander provides estimates for the optimal number of nodes to be used for gradient descent training that correspond to experimental results.

Photo of Alexander Ulanov

Alexander Ulanov

Hewlett Packard Labs

Alexander Ulanov is a senior researcher at Hewlett Packard Labs, where he focuses his research on machine learning on a large scale. Currently, Alexander works on deep learning and graphical models. He has made several contributions to Apache Spark; in particular, he implemented the multilayer perceptron classifier. Previously, he worked on text mining, classification and recommender systems, and their real-world applications. Alexander holds a PhD in mathematical modeling from the Russian Academy of Sciences.