Presented By O'Reilly and Cloudera
Make Data Work
March 13–14, 2017: Training
March 14–16, 2017: Tutorials & Conference
San Jose, CA

DevOps for models: How to manage millions of models in production

Teresa Tung (Accenture Labs), Jurgen Weichenberger (Accenture Analytics), Ishmeet Grewal (Accenture Technology Labs)
2:40pm3:20pm Thursday, March 16, 2017
Data engineering and architecture
Location: LL20 C Level: Beginner
Average rating: ***..
(3.80, 5 ratings)

Who is this presentation for?

  • Data scientists and data architects

Prerequisite knowledge

  • Experience with science in a big data environment

What you'll learn

  • Learn a DevOps approach to model management
  • Explore a specific implementation with a description of interfaces to handle new analytics engines and their versions (e.g., Spark, Python, and OpenCV)


As Accenture scaled from thousands to millions of predictive models, it needed automation to manage models at scale, ensure accuracy, prevent false alarms, and preserve trust as models are created, tested, and deployed into production. Teresa Tung, Jürgen Weichenberger, and Ishmeet Grewal discuss embracing DevOps for models by employing a self-healing approach to model lifecycle management, from model creation in the data science playground to automation for operationalizing models for decision making through monitoring their continued performance and automatically taking action when problems arise.

As with DevOps, failure is assumed, and exception handling and resiliency are considered central to the design. When data scientists submit models for deployment, they also specify a quality condition and the associated behavior when that condition is broken. Automatic retraining (self-healing) can happen as a result of breaking that condition or through regularly scheduled intervals.

Teresa, Jürgen, and Ishmeet share their approach to implementing DevOps for models and walk you through the following key steps:

  1. Code: Data scientist playground helps the data scientist create a model
  2. Build: Model hierarchy tracks composite and parent-child lineage
  3. Test: Accuracy conditions for a model and its data
  4. Package: Moving from script-based to object-based
  5. Release: Automated training, deployment into streaming
  6. Configure: Programmatic exception handling when a model is inaccurate
  7. Monitor: Continuous monitoring at scale of accuracy and health
Photo of Teresa Tung

Teresa Tung

Accenture Labs

Teresa Tung is a technology fellow at Accenture Technology Labs, where she is responsible for taking the best-of-breed next-generation software architecture solutions from industry, startups, and academia and evaluating their impact on Accenture’s clients through building experimental prototypes and delivering pioneering pilot engagements. Teresa leads R&D on platform architecture for the internet of things and works on real-time streaming analytics, semantic modeling, data virtualization, and infrastructure automation for Accenture’s industry platforms like Accenture Digital Connected Products and Accenture Analytics Insights Platform. Teresa holds a PhD in electrical engineering and computer science from the University of California, Berkeley.

Photo of Jurgen Weichenberger

Jurgen Weichenberger

Accenture Analytics

Jürgen Weichenberger is a data science senior principal at Accenture Analytics, where he is currently working within resources industries with interests in smart grids and power, digital plant engineering, and optimization for upstream industries and the water industry. Jürgen has over 15 years of experience in engineering consulting, data science, big data, and digital change. In his spare time, he enjoys spending time with his family and playing golf and tennis. Jürgen holds a master’s degree (with first-class honors) in applied computer science and bioinformatics from the University of Salzburg.

Photo of Ishmeet Grewal

Ishmeet Grewal

Accenture Technology Labs

Ishmeet Grewal is a senior research analyst at Accenture Technology Labs, where he is the lead developer responsible for developing and prototyping a comprehensive strategy for automated analytics at scale. Ishmeet has traveled to 25 countries and likes to climb rocks in his free time.