Fueling innovative software
July 15-18, 2019
Portland, OR

Democratizing AI: Making deep learning models easier to use through containerization and microservices

Saishruthi Swaminathan (IBM), IH Jhuo
3:30pm–4:05pm Tuesday, July 16, 2019
ML Ops Day
 Location: E145/146
Average rating: *****
(5.00, 1 rating)

Who is this presentation for?

  • Software engineer and developers and data scientists

Level

Beginner

Description

Nowadays, AI technologies are pervasive, especially for performance driven deep learning. Serving a deep learning model on a production system demands the understanding of the model, its reproducibility, and its ability to behave as a standalone package. One possible solution to make this simple and useful is to apply to a containerized microservice. Ideally, serving deep learning microservices should be quick and efficient, without having to dive deep into the underlying algorithms and their implementation.

Saishruthi Swaminathan and Ih Jhuo demystify the process of developing, training, and deploying deep learning models as a web microservice. You’ll get an overview of how deep learning models work; for example, TensorFlow, is best published as Docker Images on DockerHub, and is best prepared for deployment in local or cloud environments using Kubernetes or Docker. They highlight the benefits of such an approach, including standardized REST API implementation and application-friendly output format (JSON) and abstracting out the complex pre- and postprocessing portions of the model inputs and outputs, which they demonstrate with Model Asset Exchange, an open source framework developed at the IBM Center for Open Source Data and AI Technologies (CODAIT). You’ll get a walkthrough of some super cool applications such as automatic image cropping, age estimation from videos/webcam, and Veremin, a video theremin. They also discuss the challenging issue of real-time annotation for video streaming via deep learning models. Since inference process of deep learning models are time consuming, that is, comparing to generating frames from webcam device, there’s no straightforward way to annotate the sequential frames of a video in real time. You’ll also be introduced to facial age estimation deep learning model as an example to show how to solve the latency issue to annotate videos.

All these applications and the framework itself are open source, and you’re invited to contribute and enjoy the popular technologies.

Prerequisite knowledge

  • A basic understanding of microservices and Docker

What you'll learn

  • Learn how to easily deploy TensorFlow models as a web microservice and how containerization helps model serving and training
  • Understand the state-of-the-art TensorFlow deep learning models for different domains and how real-time annotation works for video streaming
Photo of Saishruthi Swaminathan

Saishruthi Swaminathan

IBM

Saishruthi Swaminathan is a developer advocate and data scientist in the IBM CODAIT team, whose main focus is to democratize data and AI through open source technologies. Her passion is to dive deep into the ocean of data, extract insights, and use AI for social good. Previously, she worked as a software developer. On a mission to spread the knowledge and experience, she acquired in her learning process. She also leads education for rural children initiative and organizing meetups focusing on women empowerment. She has a masters in electrical engineering, specializing in data science and a bachelor’s degree in electronics and instrumentation. She can be found on LinkedIn and Medium.

IH Jhuo