Model validation: When things blow up

Machine learning offers a powerful toolkit for building complex predictive systems. These models can provide immense business value and are often deployed in high-consequence environments, but it can be extremely dangerous to think of those quick wins as coming for free. Paige Bailey explains what happens when your data changes over time and fresh models must be produced continuously and details the consequences of having your model’s predictions go awry.

In the context of an energy industry case study, Paige highlights machine learning risk factors and design patterns to be avoided or refactored where possible and walks you through building and deploying a model to predict blowouts at a rigsite. You’ll learn how to make slight alterations to incoming data and to the model update strategy and see how those small changes impact the model’s accuracy. Along the way, Paige discusses common pain points for updating models in production, including boundary erosion, entanglement, hidden feedback loops, data dependencies, changes in the external world, and more, and dives into the model operationalization platform TensorFlow Extended (TFX), which provides structure for orchestrating checks against machine learning technical debt.