Presented By O’Reilly and Intel Nervana
Put AI to work
September 17-18, 2017: Training
September 18-20, 2017: Tutorials & Conference
San Francisco, CA

Deep learning on mobile: The how-to guide

Anirudh Koul (Microsoft)
1:45pm–2:25pm Wednesday, September 20, 2017
Implementing AI
Location: Yosemite A Level: Beginner
Secondary topics:  Deep learning, Tools and frameworks
Average rating: *****
(5.00, 2 ratings)

Prerequisite Knowledge

  • A high-level understanding of deep learning

What you'll learn

  • Learn how to deploy deep learning architectures on mobile devices
  • Gain practical tips on developing apps for real-world scenarios
  • Become familiar with the ecosystem and platforms available for AI on smartphones


Over the last few years, convolutional neural networks (CNN) have risen in popularity, especially in computer vision. Anirudh Koul explains how to bring the power of deep learning to memory- and power-constrained devices like smartphones, wearables, and drones.

Many mobile applications running on smartphones and wearable devices would potentially benefit from the new opportunities enabled by deep learning techniques. (An added bonus: local execution allows data to stay on the mobile device, hence avoiding latency issues of data transmission to the cloud and also alleviating privacy concerns.) However, CNNs, by nature, are computationally expensive and memory intensive, making them challenging to deploy on a mobile device. To circumvent these obstacles, Anirudh shares various strategies to build mobile-friendly shallow CNN architectures, significantly reducing the memory footprint, thus making them easier to store on a smartphone. Along the way, Anirudh explains how to build a CNN version optimized for inference on mobile devices by comparing a family of model compression techniques to prune the network size for live image processing and outlines practical strategies to preprocess your data in a manner that makes the models more efficient in the real world.

Anirudh showcases these techniques using a real-world project, discussing tips and tricks, speed and accuracy trade-offs, and benchmarks on different hardware to demonstrate how to get started developing your own deep learning application suitable for deployment on storage- and power-constrained mobile devices. Similar techniques can also be applied to make deep neural nets more efficient when deploying in a regular cloud-based production environment, reducing the number of GPUs required and optimizing on cost.

Photo of Anirudh Koul

Anirudh Koul


Anirudh Koul is a senior data scientist at Microsoft AI and Research. An entrepreneur at heart, he has been running a mini-startup team within Microsoft, prototyping ideas using computer vision and deep learning techniques for augmented reality, productivity, and accessibility, building tools for communities with visual, hearing, and mobility impairments. Anirudh brings a decade of production-oriented applied research experience on petabyte-scale social media datasets, including Facebook, Twitter, Yahoo Answers, Quora, Foursquare, and Bing. A regular at hackathons, he has won close to three dozen awards, including top-three finishes for three years consecutively in the world’s largest private hackathon, with 16,000 participants. Some of his recent work, which IEEE has called “life changing,” has been showcased at a White House AI event, Netflix, and National Geographic and to the Prime Ministers of Canada and Singapore.