Fueling innovative software
July 15-18, 2019
Portland, OR

Democratizing AI: Making deep learning models easier to use through containerization and microservices

3:30pm4:05pm Tuesday, July 16, 2019
ML Ops Day
Location: E145/146
Average rating: *****
(5.00, 1 rating)

Who is this presentation for?

  • Software engineer and developers and data scientists




Nowadays, AI technologies are pervasive, especially for performance driven deep learning. Serving a deep learning model on a production system demands the understanding of the model, its reproducibility, and its ability to behave as a standalone package. One possible solution to make this simple and useful is to apply to a containerized microservice. Ideally, serving deep learning microservices should be quick and efficient, without having to dive deep into the underlying algorithms and their implementation.

Saishruthi Swaminathan and Ih Jhuo demystify the process of developing, training, and deploying deep learning models as a web microservice. You’ll get an overview of how deep learning models work; for example, TensorFlow, is best published as Docker Images on DockerHub, and is best prepared for deployment in local or cloud environments using Kubernetes or Docker. They highlight the benefits of such an approach, including standardized REST API implementation and application-friendly output format (JSON) and abstracting out the complex pre- and postprocessing portions of the model inputs and outputs, which they demonstrate with Model Asset Exchange, an open source framework developed at the IBM Center for Open Source Data and AI Technologies (CODAIT). You’ll get a walkthrough of some super cool applications such as automatic image cropping, age estimation from videos/webcam, and Veremin, a video theremin.¬†They also discuss the challenging issue of real-time annotation for video streaming via deep learning models. Since inference process of deep learning models are time consuming, that is, comparing to generating frames from webcam device, there’s no straightforward way to annotate the sequential frames of a video in real time. You’ll also be introduced to facial age estimation deep learning model as an example to show how to solve the latency issue to annotate videos.

All these applications and the framework itself are open source, and you’re invited to contribute and enjoy the popular technologies.

Prerequisite knowledge

  • A basic understanding of microservices and Docker

What you'll learn

  • Learn how to easily deploy TensorFlow models as a web microservice and how containerization helps model serving and training
  • Understand the state-of-the-art TensorFlow deep learning models for different domains and how real-time annotation works for video streaming
Photo of Saishruthi Swaminathan

Saishruthi Swaminathan


Saishruthi Swaminathan is a developer advocate and data scientist in the IBM CODAIT team, whose main focus is to democratize data and AI through open source technologies. Her passion is to dive deep into the ocean of data, extract insights, and use AI for social good. Previously, she worked as a software developer. On a mission to spread the knowledge and experience, she acquired in her learning process. She also leads education for rural children initiative and organizing meetups focusing on women empowerment. She has a masters in electrical engineering, specializing in data science and a bachelor’s degree in electronics and instrumentation. She can be found on LinkedIn and Medium.

IH Jhuo


IH Jhuo is a software engineer/scientist in IBM. His research interests include data analysis, structured data learning algorithms, deep learning, and its applications in computer vision and multimedia. He’s been involved in designing on top performance systems and recognized by US NIST TREC video retrieval evaluation on Multimedia Event Detection task, DARPA project and the winner of the 2012 ACM Multimedia Grand Challenge 1st Place Award.