At the core of today’s problems with image classification and deep learning lies one fundamental truth: most AI systems operate by choosing the path of least resistance, not the path of maximum long-term quality. These systems collect large sets of labeled input samples and ground truth answers, train the models, evaluate the results, and let the models run. But the buck stops there.
After that, developers don’t focus on maintaining model quality. In fact, quality often regresses over time as images or inputs change. However, there’s a third option: developers can close the loop on AI by creating a running system that has access to data, in the form of submitted queries, that can improve performance.
Matt Zeiler discusses Clarifai’s approach to closing the loop on AI and the techniques it employs to counter the AI quality regression phenomenon. This includes using a feedback API allowing customers to indicate when the AI is making a mistake, having a selection of hard examples to be checked by humans or other more complex systems, and ensuring a selection of queries that can improve the system, if used for training. All of these inputs (whether using ground truth or without) can be used to improve the quality of the models via periodic updates, with minimal resources required from humans, while yielding maximum accuracy in results.
Matthew Zeiler is the founder and CEO of Clarifai, where he is applying his pioneering research in applied artificial intelligence to create developer-friendly products that allow enterprises to quickly and seamlessly integrate AI into their workflows and customer experiences. An artificial intelligence expert, Matt led groundbreaking research in computer vision, alongside renowned machine learning pioneers Geoff Hinton and Yann LeCun, that has propelled the image recognition industry from theory to real-world practice.
©2019, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org