Presented By
O’Reilly + Intel AI
Put AI to Work
April 15-18, 2019
New York, NY

Closing the Loop on AI: How to Maintain Quality Long-Term AI Results

Matt Zeiler (Clarifai)
2:40pm3:20pm Wednesday, April 17, 2019
Interacting with AI
Location: Regent Parlor
Secondary topics:  Computer Vision, Deep Learning and Machine Learning tools

Who is this presentation for?

Anyone with a basic understanding of machine learning and the challenges associated with it



Prerequisite knowledge

A basic understanding of machine learning

What you'll learn

Learn how to create a feedback loop that “closes the loop on AI” and prevents regression of image quality over time.


At the core of today’s problems with image classification and deep learning lies one fundamental truth: most AI systems operate by choosing the path of least resistance – not the path of maximum long-term quality. These systems collect large sets of labeled input samples and ground truth answers, train the models, evaluate the results and let the models run. But the buck stops there.

After that, developers don’t focus on maintaining model quality – in fact, quality often regresses over time as images or inputs change. However, there’s a third option: developers can close the loop on AI by creating a running system that has access to data, in the form of submitted queries, that can improve performance.

Matt Zeiler, founder and CEO of Clarifai, will discuss the company’s approach to Closing the Loop on AI and employing techniques to counter the AI quality regression phenomenon. Clarifai employs a variety of techniques to form a feedback loop by using the following strategies: 1. using a feedback API allowing customers to indicate when the AI is making a mistake; 2. having a selection of hard examples to be checked by humans or other more complex systems; and 3. ensuring a selection of queries that can improve the system, if used for training. All of these inputs (whether using ground truth or without) can be used to improve the quality of the models via periodic updates, with minimal resources required from humans, but yielding maximum accuracy in results.

Photo of Matt Zeiler

Matt Zeiler


Matt Zeiler, Founder and CEO of Clarifai, is a machine learning Ph.D. and thought leader pioneering the field of applied artificial intelligence (AI). Matt’s groundbreaking research in computer vision alongside renowned machine learning experts Geoff Hinton and Yann LeCun has propelled the image recognition industry from theory to real-world application. Since starting Clarifai in 2013, Matt has evolved his award-winning research into developer-friendly products that allow enterprises to quickly and seamlessly integrate AI into their workflows and customer experiences. Today, Clarifai is the leading independent AI company and “widely seen as one of the most promising [startups] in the crowded, buzzy field of machine learning.” (Forbes) Reach him @MattZeiler.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)