Deploying machine learning models on the edge

Yan Zhang (Microsoft), Mathew Salvaris (Microsoft)

11:05–11:45 Wednesday, 16 October 2019

Location: King's Suite - Sandringham

Implementing AI

Secondary topics: Hardware, Machine Learning tools, Mobile Computing, IoT, Edge

Average rating:

(3.67, 3 ratings)

Download slides (PDF)

Who is this presentation for?

Data scientists

Level

Intermediate

Description

When IoT meets AI, a new round of innovations on big data, cloud computing, and intelligent edge begins.

By 2020, it’s estimated that 250 petabytes of data will be generated by personal or enterprise IoT devices every day. Edge computing is well suited to handle this data, because it provides a means to collect and process data at local computing devices rather than in the cloud or a remote data center. It has two key benefits to IoT applications: a real-time analysis of data and reduced data transmission to the cloud. Therefore, IoT devices incur less latency and react more quickly to changes in status. As part of edge computing, intelligent edge aims to bring predictive analytics on the edge devices.

Yan Zhang and Mathew Salvaris explore the methodology, practice, and tools around deploying machine learning models on the edge, offering a step-by-step guide to creating a pretrained ML model using Python, packaging it in a Docker container, and deploying it as a local service on an edge device. They outline how to test and verify each step and discover the gotchas you may encounter. You’ll see a demo of how to make calls to the deployed service to make predictions on a predeployed edge device. Yan and Mathew also discuss the consideration of deployment on GPU-enabled edge devices as well as how the edge devices can be managed in a centralized way in the cloud. Such a strategy makes it easy to train and retrain ML models in the cloud and to deploy the trained models on the multiple edge devices at the same time.

Prerequisite knowledge

Familiarity with machine learning and Python

What you'll learn

Learn how to build a pretrained machine learning model into a Docker image, how to deploy machine models on edge devices through Docker and containerization, and how the machine learning model can be managed and retrained in the cloud and deployed on multiple edge devices
Understand GPU versus CPU edge device deployment through an example using Azure IoT Edge

Yan Zhang

Microsoft

Yan Zhang is a senior data scientist with the algorithm and data science team of the Data Group within Cloud and Enterprise at Microsoft. She builds predictive analytics models and generalizes machine learning solutions on the cloud machine learning platform. Her recent research includes cost prediction and fraud claim detection in the healthcare domain, predictive maintenance in IoT applications, customer segmentation, and text mining. Previously, she was a research faculty member at Syracuse University. Yan earned her PhD in data mining from the Computer Science Department at the University of Vermont. She’s the author of 23 publications, including journal articles, conference papers, and blog posts. Her first paper won the best paper award at the 17th IEEE International Conference on tools with artificial intelligence. She’s one of the reviewers for the book Predictive Analytics with Microsoft Azure Machine Learning, second edition, published in September 2015.

Mathew Salvaris

Microsoft

Mathew Salvaris is a data scientist at Microsoft. Previously, Mathew was a data scientist for a small startup that provided analytics for fund managers; a postdoctoral researcher at UCL’s Institute of Cognitive Neuroscience, where he worked with Patrick Haggard in the area of volition and free will, devising models to decode human decisions in real time from the motor cortex using electroencephalography (EEG); and a postdoc in the University of Essex’s Brain Computer Interface Group, where he worked on BCIs for computer mouse control. Mathew holds a PhD in brain-computer interfaces and an MSc in distributed artificial intelligence.