Fueling innovative software
July 15-18, 2019
Portland, OR

Model as a service for real-time decisioning​

Niraj Tank (Capital One), Sumit Daryani (Capital One)
9:20am9:55am Tuesday, July 16, 2019
ML Ops Day
Location: E145/146
Average rating: *****
(5.00, 2 ratings)

Who is this presentation for?

  • Architects, technology leaders, and data scientists
  • Product managers seeking information to operationalizing machine learning models




Hosting models and productionizing them is a pain point. Let’s fix that.

Imagine a stream processing platform that leverages ML models and requires real-time decisions. While most solutions provide tightly coupled ML models in the use case, these may not offer the most efficient way for a data scientist to update or roll back a model. With model as a service, disrupting the flow and relying on technical engineering teams to deploy, test, and promote their models is a thing of the past. It’s time to focus on building a decoupled service-based architecture while upholding engineering best practices and delivering gains in terms of model management and deployment. Other benefits also include empowering data scientists by supporting patterns such as A/B testing, multiarm bandits, and ensemble modeling.

Niraj Tank and Sumit Daryani demonstrate their work with a reference architecture implementation for building the set of microservices and lay down, step by step the critical aspects of building a well-managed ML model deployment flow pipeline that requires validation, versioning, auditing, and model risk governance. They discuss the benefits of breaking the barriers of a monolithic ML use case by using a service-based approach consisting of features, models, and rules. Join in to gain insights into the technology behind the scenes that accepts a raw serialized model built using popular libraries like H2O, scikit-learn, or TensorFlow, or even plain Python source models and serve them via REST/gRPC which makes it easy for the models to integrate into business applications and services that need predictions.

Prerequisite knowledge

  • A basic understanding of microservices and container orchestration technologies like Kubernetes

What you'll learn

  • Learn to build an ML pipeline for decisioning using microservices, making use of open source container technologies
  • Understand DevOps practices that involve agility, automated QA, rolling upgrades, one-click promotion, and fully automated deployments on a real-time decisioning platform
Photo of Niraj Tank

Niraj Tank

Capital One

Niraj Tank is a senior manager, software engineer at Capital One working on a team that built a fast data streaming and decisioning platform for Capital One. Niraj has been an engineer for the past 21 years, and his diverse experience ranges from developing products for startups to leading various large-scale integration services.

Photo of Sumit Daryani

Sumit Daryani

Capital One

Sumit Daryani is a software engineering manager and architect at Capital One. He works on a real-time machine learning decision platform to protect its banking platform and foster quick decisions to support the fraud strategy. Previously, Sumit was a full-stack engineer on a diverse number of projects scaling from the financial to the technology space.