Mar 15–18, 2020

ML models are not software: Why organizations need dedicated operations to address the b

Anand Rao (PwC), Joseph Voyles (PwC)
4:15pm4:55pm Wednesday, March 18, 2020
Location: LL21 D

Who is this presentation for?

  • Enterprise technology or data leaders tasked with developing, scaling, and driving adoption of an enterprise AI capability; data science practitioners seeking to move models from enabling analysis to a broader set of enterprise assets




You’ve developed a best-in-class data science team, created a cutting-edge model storage and compute infrastructure, and adopted leading data science practices. However, the investment hasn’t yielded the expected returns. You’ve only been able to deliver ML solutions to a small fraction of your enterprise, and you’re increasingly balancing new model development with an emerging and unexpected workload that is formed by the need to support continuous learning, technical adaptation, and evolving risks. This increased workload is due to many challenges, including:

  • Models aren’t like software: Models are never complete; they’re a product of experimentation. New techniques emerge for improvement and new data can enhance performance.
  • Continuously moving performance: Model performance will never be static in a production setting, and remediating performance issues requires significant resource investment compared to traditional software.
  • A technical ecosystem in search of equilibrium: Intelligent systems are enabled by a complex ecosystem of homegrown and third-party compute, data, development tools, and models that are constantly evolving and being replaced.
  • High-stakes predictions: Models support complex decisions and automations that cannot be accomplished through traditional software; this creates risk exposure and on-going overhead to manage.

As AI grows into its role of a chief enabling technology, machine learning models will find their way into a more diverse set of digital assets that, includes automation workflows, bots, traditional applications, and devices. As the operational challenges mount, organizations will begin to lean on existing development operations practices. However, the lifecycle of software and machine learning models offers a set of fundamental differences that limit their effectiveness, and a new approach is required.

Anand Rao and Joseph Voyles believe that these AI adoption and scaling challenges must be addressed through a dedicated AI operations capability. Borrowing from existing DevOps tools is an option; however, these challenges cannot be addressed through just technology or frameworks designed for managing software. They propose an AI operations function that addresses the limits of scale through the following capabilities:

  • Governance: A digital asset that constantly changes requires a governance structure that maintains pace while still maintaining trust and safety.
  • Process: Automate everything, but within reason. Deployment, performance monitoring, retraining, and model, data, and algorithm versioning should all have automated elements. However, human oversight must remain, especially in systems where models perform critical functions.
  • Operations philosophy: Model development, deployment infrastructure, and operations practices must be designed and developed to optimize the efficiency and productivity of a data science team. Software development agile methodologies need to be adapted to incorporate the test-and-learn philosophy of model development.
  • Technology and infrastructure: Models change. Every layer of the technology stack from development to deployment and monitoring must support an ability to track, record, and automate change.
  • A flexible architecture: An AI operations architecture needs to be able to scale and enable to the virtuous cycle of AI while offering the flexibility to adapt to a dynamic technology landscape.

Anand and Joseph also detail industry examples that highlight the proposed production ML scaling challenges, architectures for creating a continuous learning platform, and creating integrated model development and deployment environment.

Prerequisite knowledge

  • General knowledge of machine learning fundamentals, development operations, and data and model governances

What you'll learn

  • Recognize the primary differences between software and model development and delivery lifecycles, the challenges these differences create when developing AI, and the dynamics limiting your ability to scale adoption to generate value from AI
  • Understand the need to build AIOps and the architectural, organizational, process, and technical best practices that will help your overcome these challenges with scale
Photo of Anand Rao

Anand Rao


Anand Rao is a partner in PwC’s Advisory Practice and the innovation lead for the Data and Analytics Group, where he leads the design and deployment of artificial intelligence and other advanced analytical techniques and decision support systems for clients, including natural language processing, text mining, social listening, speech and video analytics, machine learning, deep learning, intelligent agents, and simulation. Anand is also responsible for open source software tools related to Apache Hadoop and packages built on top of Python and R for advanced analytics as well as research and commercial relationships with academic institutions and startups, research, development, and commercialization of innovative AI, big data, and analytic techniques. Previously, Anand was the chief research scientist at the Australian Artificial Intelligence Institute; program director for the Center of Intelligent Decision Systems at the University of Melbourne, Australia; and a student fellow at IBM’s T.J. Watson Research Center. He has held a number of board positions at startups and currently serves as a board member for a not-for-profit industry association. Anand has coedited four books and published over 50 papers in refereed journals and conferences. He was awarded the most influential paper award for the decade in 2007 from Autonomous Agents and Multi-Agent Systems (AAMAS) for his work on intelligent agents. He’s a frequent speaker on AI, behavioral economics, autonomous cars and their impact, analytics, and technology topics in academic and trade forums. Anand holds an MSc in computer science from Birla Institute of Technology and Science in India, a PhD in artificial intelligence from the University of Sydney, where he was awarded the university postgraduate research award, and an MBA with distinction from Melbourne Business School.

Photo of Joseph Voyles

Joseph Voyles


Joseph Voyles is a director at PwC.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

For conference registration information and customer service

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

For media/analyst press inquires