Presented By O'Reilly and Cloudera
Make Data Work
September 26–27, 2016: Training
September 27–29, 2016: Tutorials & Conference
New York, NY

Stream analytics in the enterprise: A look at Intel’s internal IoT implementation

Moty Fania (Intel)
4:35pm–5:15pm Thursday, 09/29/2016
IoT & real-time
Location: 1 E 12/1 E 13 Level: Intermediate

Prerequisite knowledge

  • A basic knowledge of big data technologies and the architecture of stream analytics systems
  • What you'll learn

  • Understand the set of characteristics common to many IoT scenarios that are required in a reusable platform
  • Learn the architecture and implementation of Intel's internal IoT platform
  • Discover how an IoT platform can be used to address problems beyond classical IoT use cases
  • Description

    Recent years have seen significant evolution of the Internet of Things. It has become increasingly easy to connect devices to the Internet and send sensory data to the public cloud. However, the adoption of IoT platforms and stream analytics within the enterprise is lagging and less prevalent, an effect of the lack of skilled developers required to deploy an on-premises platform and the limited demonstration of high value in real-life use cases.

    Intel IT has addressed these challenges by implementing an internal IoT platform, with the goal of allowing users and organizations to gain insights and business value from real-time analytics. The platform is based on several open source technologies including Akka, Kafka, and Spark Streaming with a full stack of algorithms including multisensor change detection and anomaly detection. To enable stream analytics at scale, Intel implemented a smart data pipe/stream processing framework, Pigeon, that implements a cluster capable of processing topologies that process the data according to any arbitrary logic determined by the users and is optimized to be easily deployed with Docker and CoreOS, which cuts down development by enabling a single developer to deploy a massive real time, elastic processing cluster with a click of a button. And unlike other IoT analytics implementations that settle for basic statistics or make many assumptions on the collected data, Intel implemented a generic analytics layer that uses machine learning and advanced statistical tests to provide meaningful insights to users in different use cases and business domains.

    Moty Fania explains how Intel identified the set of characteristics and needs common to many IoT scenarios and made them available in one single reusable platform, offers a thorough overview of the platform’s architecture and related technologies (Akka, Kafka, Spark, Hadoop, etc.), demonstrates how Docker and CoreOS made the on-premises deployment easy, and reviews the generic analytics layer that uses machine learning to provide meaningful insights in different use cases and business domains. Moty concludes by discussing how Intel is using this platform to address problems that are not classical IoT use cases but can benefit from real-time analytics to achieve proactivity and operational excellence.

    Photo of Moty Fania

    Moty Fania


    Moty Fania is a principal engineer and the CTO of the Advanced Analytics Group at Intel, which delivers AI and big data solutions across Intel. Moty has rich experience in ML engineering, analytics, data warehousing, and decision-support solutions. He led the architecture work and development of various AI and big data initiatives such as IoT systems, predictive engines, online inference systems, and more.

    Comments on this page are now closed.


    Picture of Moty Fania
    Moty Fania
    09/29/2016 6:34pm EDT

    The slides are now uploaded. Thanks :)

    Igor Vasilchikov
    09/29/2016 5:25pm EDT

    how to get slides for the talk ?