Presented By O’Reilly and Cloudera
Make Data Work
September 11, 2018: Training & Tutorials
September 12–13, 2018: Keynotes & Sessions
New York, NY

Using machine learning to drive intelligence at the edge

Dave Shuman (Cloudera), Bryan Dean (Red Hat)
4:35pm–5:15pm Wednesday, 09/12/2018
Data engineering and architecture
Location: 1E 09 Level: Intermediate
Secondary topics:  Model lifecycle management

Who is this presentation for?

  • Solution architects, big data professionals, IoT leads, IoT and data architects, and line of business managers and directors who are thinking about deploying IoT use cases

Prerequisite knowledge

  • A basic understanding of the IoT, data analytics, and the relevance of their key open source components

What you'll learn

  • Explore an end-to-end reference architecture for the IoT and lessons learned deploying this architecture
  • Learn how to drive more intelligence to the edge
  • See how this was implemented in one of the leading manufacturers in Europe to potentially detect fire using image recognition
  • Understand how to leverage various components within the open source ecosystem to enable IoT use cases


With 30+ billion connected things by 2020, the IoT will drive an explosion of data that will need to be processed, stored, managed, and analyzed—in most cases, in real time—to derive business value. However, the volume, diversity, speed, and inherent characteristics of data generated from sensors and the IoT will challenge traditional data management mechanisms. Given this, IoT architectures are increasingly focused on making the edge more intelligent in an effort to lower round-trip latencies and minimize cost of data transmission.

More importantly, machine learning is starting to play a key role in enabling IoT use cases today. Organizations need to be able to do advanced analytics to enable concepts such as pattern recognition, anomaly detection, and ultimately predictive modeling based on the petabytes of data that the IoT generates. And this is where large-scale machine learning and advanced analytics comes into play.

So how can organizations utilize machine learning, deep learning, and advanced analytics to make intelligent decisions closer to where data is generated? How can you build machine learning models for the IoT and push those out back to the edge? What role do open source technologies play in this end-to-end architecture?

The way to make the edge more intelligent is by building machine learning models in a centralized hub in the cloud and push the knowledge generated out back to the edge. The possibilities for use cases utilizing machine learning at the edge are endless.

Dave Shuman and Bryan Dean explore a recent proof of concept (PoC) that was executed at one of Europe’s leading global manufacturers. Cloudera, Red Hat, and Eurotech have deployed an end-to-end architecture, built on open source technologies, for the IoT that enables end-to-end analytics, including business rules and advanced analytical models that can be deployed both at the edge and within the centralized data platform. They were able to train a deep learning image recognition model of fire and deploy onto a constrained IoT edge gateway device, where it could evaluate if fire broke out in a factory floor in real time. They were able to demonstrate edge model execution for a machine learning model using Eclipse Deeplearning4j deployed as an OSGi bundle. Join Dave and Bryan to learn more about this project and discover how organizations are using advanced analytics and machine learning to make the edge more intelligent. Along the way, Dave and Bryan walk you through a roadmap illustrating how organizations can utilize machine learning and effectively push that knowledge back out to the edge. They then detail an end-to-end open source architecture for the IoT based on Eclipse Kura, an open source stack for gateways and the edge, and Eclipse Kapua, an open source IoT cloud platform. The architecture can enable:

  • Capabilities to securely connect and manage millions of distributed IoT devices and gateways
  • Machine learning and analytics capabilities with intelligence and analytics at the edge
  • A centralized data management and analytics platform with the ability to build or refine machine learning models and push them out to the edge
  • Application development, deployment, and integration services

They conclude with an Industry 4.0 demo that highlights how to ingest, process, and analyze data coming off of factory equipment and how to enable machine learning on the edge using all of this data.

Photo of Dave Shuman

Dave Shuman


Dave Shuman is the industry lead for the IoT and manufacturing at Cloudera. Dave has an extensive background in big data analytics, business intelligence applications, database architecture, logical and physical database design, and data warehousing. Previously, Dave held a number of roles at Vision Chain, a leading demand signal repository provider enabling retailer and manufacturer collaboration, including chief operations officer, vice president of field operations responsible for customer success and user adoption, vice president of product responsible for product strategy and messaging, and director of services. He also served at such top CG companies as Kraft Foods, PepsiCo, and General Mills, where he was responsible for implementations; was vice president of operations for enews, an ecommerce company acquired by Barnes and Noble; was executive vice president of management information systems, where he managed software development, operations, and retail analytics; and developed ecommerce applications and business processes used by, Yahoo, and Excite and pioneered an innovative process for affiliate commerce. He holds an MBA with a concentration in information systems from Temple University and a BA from Earlham College.

Photo of Bryan Dean

Bryan Dean

Red Hat

Bryan Dean is director of business development at Red Hat, where he is leading the company’s global initiatives and solutions for the internet of things (IoT) within Red Hat’s Global Partner Solutions Group. As the business development lead for the IoT, Bryan is responsible for solution development, partnerships, and go-to-market initiatives. Working extensively with key strategic partners Cloudera and Eurotech, Bryan has led the effort to develop and promote an end-to-end enterprise architecture for the IoT built on open source technologies. The architecture was awarded IoT Infrastructure of the Year during Computing’s Big Data Awards in May, 2018. Previously, Bryan held leadership positions at NetApp and Hewlett-Packard Software. His background includes product management, marketing, alliances, and strategic business planning. Bryan is originally from the San Francisco Bay Area but has called Fort Collins, Colorado, home for the last 20 years.