Presented By O’Reilly and Cloudera
Make Data Work
March 5–6, 2018: Training
March 6–8, 2018: Tutorials & Conference
San Jose, CA

Explaining machine learning models

Evan Kriminger (ZestFinance)
11:00am11:40am Thursday, March 8, 2018
Average rating: ****.
(4.40, 5 ratings)

Who is this presentation for?

  • Data scientists, analysts, and other stakeholders in model deployment

What you'll learn

  • Understand the strengths and limitations of leading approaches to model explainability and how explainability approaches can solve critical business problems, such as model validation


Machine learning models are often complex, with massive abstract descriptions that make the relationship between their inputs and outputs seem like a black box. A modern neural network, for example, might look at thousands of features and perform millions of additions and multiplications to produce a prediction. But how do we explain that prediction to someone else? How do we tell which features are important and why? And if we can’t understand how a model makes a prediction, do we really trust it to run our business, make medical conclusions, or make an unbiased decision about an applicant’s eligibility for a loan?

Explainability techniques clarify how models make decisions, offering answers to these questions and giving us confidence that our models are functioning properly (or not). Each of these techniques is applicable to a different set of models, makes different assumptions, and answers a slightly different question, but when used properly, they can meet business requirements and improve model performance.

Mike Ruberry offers an overview of the two main types of explainability. The first directly relates inputs to outputs, a naturally intuitive approach that includes local interpretable model-agnostic explanations (LIME), axiomatic attributions, VisualBackProp, and traditional feature contributions. The second makes use of the data the model was trained on. DeepLift, for example, can show which training examples were most relevant to a model’s decision, while scrambling and prototype methods detail the decision making process. Along the way, Mike covers how ZestFinance approaches explainability, offering a practical guide for your own work. While there is no perfect silver-bullet explainability technique, understanding when and how to use these approaches will let you explain many useful models and give you a broad view of current explainability best practices and research.

Photo of Evan Kriminger

Evan Kriminger


Evan Kriminger is a Senior Associate of Data Science at ZestFinance, where his research interests include explainability and building efficient tools for training deep neural networks. He holds a PhD from the Computational NeuroEngineering Laboratory at the University of Florida, completing a dissertation on active learning and constrained clustering. Prior to ZestFinance, he worked at Leap Motion, conducting machine learning research for hand tracking.