Put AI to Work
April 15-18, 2019
New York, NY
Please log in

Closing the loop on AI: How to maintain quality long-term AI results

Matt Zeiler (Clarifai)
2:40pm3:20pm Wednesday, April 17, 2019
Interacting with AI
Location: Regent Parlor
Secondary topics:  Computer Vision, Deep Learning and Machine Learning tools



Prerequisite knowledge

  • A basic understanding of machine learning

What you'll learn

  • Learn how to create a feedback loop that “closes the loop on AI” and prevents regression of image quality over time


At the core of today’s problems with image classification and deep learning lies one fundamental truth: most AI systems operate by choosing the path of least resistance, not the path of maximum long-term quality. These systems collect large sets of labeled input samples and ground truth answers, train the models, evaluate the results, and let the models run. But the buck stops there.

After that, developers don’t focus on maintaining model quality. In fact, quality often regresses over time as images or inputs change. However, there’s a third option: developers can close the loop on AI by creating a running system that has access to data, in the form of submitted queries, that can improve performance.

Matt Zeiler discusses Clarifai’s approach to closing the loop on AI and the techniques it employs to counter the AI quality regression phenomenon. This includes using a feedback API allowing customers to indicate when the AI is making a mistake, having a selection of hard examples to be checked by humans or other more complex systems, and ensuring a selection of queries that can improve the system, if used for training. All of these inputs (whether using ground truth or without) can be used to improve the quality of the models via periodic updates, with minimal resources required from humans, while yielding maximum accuracy in results.

Photo of Matt Zeiler

Matt Zeiler


Matthew Zeiler is the founder and CEO of Clarifai, where he is applying his pioneering research in applied artificial intelligence to create developer-friendly products that allow enterprises to quickly and seamlessly integrate AI into their workflows and customer experiences. An artificial intelligence expert, Matt led groundbreaking research in computer vision, alongside renowned machine learning pioneers Geoff Hinton and Yann LeCun, that has propelled the image recognition industry from theory to real-world practice.