Presented By O'Reilly and Cloudera
Make Data Work
March 13–14, 2017: Training
March 14–16, 2017: Tutorials & Conference
San Jose, CA

Scalable deep learning for the enterprise with DL4J

Dave Kale (Skymind), Susan Eraly (Skymind), Josh Patterson (Skymind)
1:30pm5:00pm Tuesday, March 14, 2017
Data science & advanced analytics
Location: LL20 D Level: Intermediate
Secondary topics:  Deep learning
Average rating: ***..
(3.33, 3 ratings)

Who is this presentation for?

  • Data scientists, data engineers, and machine-learning engineers at all organizational levels

Prerequisite knowledge

  • Familiarity with basic machine-learning terminology and concepts (e.g., classification)
  • Experience with Java and Maven (You must be able to use Maven to build a Java project without assistance.)

Materials or downloads needed in advance

  • A laptop with Java (developer version) 1.7 or later (64-bit only), Apache Maven (automated build and dependency manager), IntelliJ IDEA or Eclipse, and Git installed (Follow the instructions in the DL4J Quick Start Guide and the Comprehensive Setup Guide—a Docker image with all required software available for download in advance of the tutorial will be made available.)
  • A high-performance computing library, such as MKL, OpenBLAS, or CUDA, as described on the Building DL4J Locally page (useful but not required)

What you'll learn

  • Learn how to leverage big data to solve real-world problems using deep learning
  • Understand how to formulate real-world prediction problems as machine-learning tasks, how to choose the right neural net architecture for the problem, and how to train neural nets using DL4J

Description

Deep learning offers a powerful suite of techniques for building predictive models and creating value from large stores of digital data that are increasingly common in domains as diverse as manufacturing, finance, advertising, and healthcare. Deep learning models have achieved near or better than human performance in machine translation, speech recognition, and image classification and recently beat the world’s best human Go players, a feat that was believed to be decades away.

The rise of deep learning has several implications for data mining and analytics. First, building successful predictive models depends less on designing good features than it does upon matching architectures to problems and tuning hyperparameters. These in turn require a new set of skills and intuitions among practitioners. Second, deep learning models are computationally expensive to train and validate. Training even simple models on modestly sized datasets can require specialized hardware like GPUs while state-of-the-art performance featured in popular press often involves training large models with massive datasets on complex distributed compute platforms.

Dave Kale, Susan Eraly, and Josh Patterson provide a practical introduction to training neural networks using Deeplearning4j (DL4J), the open source, Java-based deep learning framework. Through both presented material and hands-on exercises, you’ll learn how to meet the challenges that arise when you apply deep learning to your business problems as Dave, Melanie, Susan, and Josh walk you through using DL4J to build and train several canonical neural net architectures, including a convolutional network for image classification and a recurrent neural net for sequence modeling.

Topics include:

  • A review of key machine-learning terminology and fundamentals, including task formulation, principled evaluation, overfitting and regularization, loss functions, performance metrics, etc.
  • The building blocks of deep learning: Gradient descent, backpropagation, nonlinearities, dropout, etc.
  • Specialized neural net architectures for structured data, such as convolutions and recurrent layers
  • An overview of the DL4J ecosystem: ND4J, for NumPy-like arrays and operations, DataVec, for data preparation pipelines, Deeplearning4j for building neural net architectures, RL4J for reinforcement learning, Arbiter for hyperparameter tuning, and ScalNet, a Keras-like Scala API
  • Parallel training with multiple GPUs and distributed training via the DL4J Spark API
  • Loading pretrained models from other frameworks, such as Keras
Photo of Dave Kale

Dave Kale

Skymind

David Kale is a deep learning engineer at Skymind and a PhD candidate in computer science at the University of Southern California (advised by Greg Ver Steeg of the USC Information Sciences Institute). David’s research uses machine learning to extract insights from digital data in high-impact domains, such as healthcare. Recently, he has pioneered the application of recurrent neural nets to modern electronic health records data. At Skymind, he is developing the ScalNet Scala API for DL4J and working on model interoperability between DL4J and other major frameworks. David organizes the Machine Learning and Healthcare Conference (MLHC), is a cofounder of Podimetrics, and serves as a judge in the Qualcomm Tricorder XPRIZE competition. David is supported by the Alfred E. Mann Innovation in Engineering Fellowship.

Photo of Susan Eraly

Susan Eraly

Skymind

Susan Eraly is a software engineer at Skymind, where she contributes to Deeplearning4j. Previously, Susan worked as a senior ASIC engineer at NVIDIA and as a data scientist in residence at Galvanize.

Photo of Josh Patterson

Josh Patterson

Skymind

Josh Patterson is the director of field engineering for Skymind. Previously, Josh ran a big data consultancy, worked as a principal solutions architect at Cloudera, and was an engineer at the Tennessee Valley Authority, where he was responsible for bringing Hadoop into the smart grid during his involvement in the openPDC project. Josh is a graduate of the University of Tennessee at Chattanooga with a master of computer science, where he did research in mesh networks and social insect swarm algorithms. Josh is a cofounder of the DL4J open source deep learning project and is a coauthor on the upcoming O’Reilly title Deep Learning: A Practitioner’s Approach. Josh has over 15 years’ experience in software development and continues to contribute to projects such as DL4J, Canova, Apache Mahout, Metronome, IterativeReduce, openPDC, and JMotif.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Comments

Nirav Patel | SENIOR SOFTWARE ENGINEER
03/15/2017 9:17am PDT

hi, can you please share the material for this session.