Fueling innovative software
July 15-18, 2019
Portland, OR

What's your machine learning score?

Tania Allard (Microsoft)
5:05pm5:45pm Thursday, July 18, 2019
Secondary topics:  AI Enhanced
Average rating: *****
(5.00, 2 ratings)

Who is this presentation for?

  • Data scientists, software engineers, DevOps, and infrastructure engineers




Using ML in real-world applications and production systems is a very complex task involving issues rarely encountered in toy problems, R&D environments, or offline cases. Key considerations for accessing the decay, current status, and production readiness of ML systems include testing, monitoring, and logging, but how much is enough? It’s difficult to know where to get started or even to know who should be responsible for the testing and monitoring. If you’ve heard the phrase “test in production” too often when it comes to ML, perhaps you need to change your strategy.

Tania Allard dives deep into some of the most frequent issues encountered in real-life ML applications and how you can make your systems more robust, and she explores a number of indicators pointing to decay of models or algorithms in production systems. Some of the topics covered include problems and pitfalls of ML in production; introducing a rubric to test and monitor your ML applications; and testing data and features, testing your model development, monitoring your ML applications, and model decay.

You’ll leave with a clear rubric with actionable tests and examples to ensure the quality or model in production is adequate. Engineers, DevOps, and data scientists will gain valuable guidelines to evaluate and improve the quality of their ML models before anything reaches production stage.

Prerequisite knowledge

  • A basic knowledge of ML (as a data scientist, an engineer developing tools for ML applications or applications, or as an infrastructure engineer working in data intensive applications)

What you'll learn

  • Learn different approaches and points to test before deploying ML applications in production and approaches to testing complex data pipelines and your infrastructure readiness
  • Understand the concept of model and ML learning decay and why it's important to detect this on time (and what to do about it)
  • Photo of Tania Allard

    Tania Allard


    Tania Allard (she/her) is a cloud developer advocate at Microsoft and a research engineer with vast experience in academic research and industrial environments. Her main areas of expertise are within data-intensive applications, scientific computing, and machine learning; one of her main areas is the improvement of processes, reproducibility, and transparency in research, data science, and artificial intelligence. Over the last few years, she’s trained hundreds of people on scientific computing, reproducible workflows, and ML models testing, monitoring, and scaling and delivered talks on the topic worldwide. She’s passionate about mentoring, open source and its community, and she’s involved in a number of initiatives aimed to build more diverse and inclusive communities. She’s also a contributor, maintainer, and developer of a number of open source projects and the Founder of Pyladies NorthWest UK.