Presented By O’Reilly and Cloudera
Make Data Work
21–22 May 2018: Training
22–24 May 2018: Tutorials & Conference
London, UK

A high-performance system for deep learning inference and visual inspection

Moty Fania (Intel)
14:0514:45 Thursday, 24 May 2018

Who is this presentation for?

Developers and data scientists

Prerequisite knowledge

None

What you'll learn

If you are planning to implement a similar platform to benefit from the value of visual inference / inspection, you will get a chance to learn from our experience: • How we identified the set of characteristics and needs that are common to AI scenarios and made them available in this platform. • Thorough overview of the architecture we implemented with the related technologies (TensorFlow serving, Redis, Flask etc.) • How Docker and Kubernetes made the on premise deployment easy • Explain potential use cases that can leverage deep learning visual inference to provide meaningful insights in different scenarios. • Hear how we are using this platform to address visual inspection use cases that are essential to accelerate various product development and validation processes at Intel

Description

Recent years have seen significant evolvement of deep learning and AI capabilities. AI solutions can augment or replace mundane tasks, increase workforce productivity and relief human bottlenecks. Unlike traditional automation, these solutions have cognitive aspects which used to require human decision making. In some cases, deep Learning has proven to be even more accurate than humans in identifying patterns and therefore can be effectively used to enable various kinds of automated, real- time decision making.

The advanced Analytics team at Intel IT has addressed these needs and implemented an internal visual inference platform – a high-performance system for deep learning inference, designed for production environments. This innovative system enables easy deployment of many DL models in production while enabling a closed feedback loop were data flows in and decisions are returned through a fast REST API. The system maximizes throughputs through batching and smart in-memory caching and can be deployed either as a cluster or standalone node.

To enable stream analytics at scale, the system was built in a modern micro-services architecture using cutting edge technologies such as TensorFlow, TensorFlow serving, Redis, Flask and more. It is optimized to be easily deployed with Docker and Kubernetes and to cut down time to market for deploying a DL solution. By supporting different kinds of models and various inputs including images and video streams this system can enable deployment of smart visual inspection solutions with real- time decision making.

If you are planning to implement a similar platform to benefit from the value of visual inference / inspection, you will get a chance to learn from our experience:
• How we identified the set of characteristics and needs that are common to AI scenarios and made them available in this platform.
• Thorough overview of the architecture we implemented with the related technologies (TensorFlow serving, Redis, Flask etc.)
• How Docker and Kubernetes made the on premise deployment easy
• Explain potential use cases that can leverage deep learning visual inference to provide meaningful insights in different scenarios.
• Hear how we are using this platform to address visual inspection use cases that are essential to accelerate various product development and validation processes at Intel

Impact : The presented platform and analytic capabilities were applied to several use cases at Intel and demonstrated excellent results. For example, in one use case it was able to successfully detect wafer defects based on optic microscope images of each layer. In another use case, it was used to replace a very tedious and costly human effort of inspecting video / game rendering produced by Intel’s new GPU products. In addition, we found that this capability opens the door for additional high value DL solutions that require large scale inference in production.

Photo of Moty Fania

Moty Fania

Intel

Moty Fania owns development and architecture in the advanced analytics group within Intel IT. With over 13 years of experience in analytics, data warehousing, and decision support solutions, Moty drives the overall technology and architectural roadmap for big data analytics in Intel IT. Moty is also the architect behind Intel’s IoT big data analytics platform. He holds a bachelor’s degree in computer science and economics and a master’s degree in business administration from Ben-Gurion University in Israel.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)