Put open source to work
July 16–17, 2018: Training & Tutorials
July 18–19, 2018: Conference
Portland, OR

Model validation: When things blow up

Paige Bailey (Microsoft)
2:00pm2:30pm Tuesday, July 17, 2018
Location: B115-116
Tags: tensorflow
Average rating: *****
(5.00, 1 rating)

Machine learning offers a powerful toolkit for building complex predictive systems. These models can provide immense business value and are often deployed in high-consequence environments, but it can be extremely dangerous to think of those quick wins as coming for free. Paige Bailey explains what happens when your data changes over time and fresh models must be produced continuously and details the consequences of having your model’s predictions go awry.

In the context of an energy industry case study, Paige highlights machine learning risk factors and design patterns to be avoided or refactored where possible and walks you through building and deploying a model to predict blowouts at a rigsite. You’ll learn how to make slight alterations to incoming data and to the model update strategy and see how those small changes impact the model’s accuracy. Along the way, Paige discusses common pain points for updating models in production, including boundary erosion, entanglement, hidden feedback loops, data dependencies, changes in the external world, and more, and dives into the model operationalization platform TensorFlow Extended (TFX), which provides structure for orchestrating checks against machine learning technical debt.

Photo of Paige Bailey

Paige Bailey


Paige Bailey is a senior cloud developer advocate at Microsoft specializing in machine learning and artificial intelligence. Previously, Paige was a data scientist and machine learning engineer in the energy industry (drilling and completions optimization, subsurface characterization). Paige has over a decade of experience doing data analysis with Python and five years of building predictive models with R. She serves on the core committee for JupyterCon and SciPy, is a Python instructor for EdX, founded PyLadies-HTX in Houston, and is currently writing both an introductory children’s book on machine learning and a technical cookbook for machine learning at scale with tools like Apache Spark.