Recent years have seen significant evolution of the Internet of Things. It has become increasingly easy to connect devices to the Internet and send sensory data to the public cloud. However, the adoption of IoT platforms and stream analytics within the enterprise is lagging and less prevalent, an effect of the lack of skilled developers required to deploy an on-premises platform and the limited demonstration of high value in real-life use cases.
Intel IT has addressed these challenges by implementing an internal IoT platform, with the goal of allowing users and organizations to gain insights and business value from real-time analytics. The platform is based on several open source technologies including Akka, Kafka, and Spark Streaming with a full stack of algorithms including multisensor change detection and anomaly detection. To enable stream analytics at scale, Intel implemented a smart data pipe/stream processing framework, Pigeon, that implements a cluster capable of processing topologies that process the data according to any arbitrary logic determined by the users and is optimized to be easily deployed with Docker and CoreOS, which cuts down development by enabling a single developer to deploy a massive real time, elastic processing cluster with a click of a button. And unlike other IoT analytics implementations that settle for basic statistics or make many assumptions on the collected data, Intel implemented a generic analytics layer that uses machine learning and advanced statistical tests to provide meaningful insights to users in different use cases and business domains.
Moty Fania explains how Intel identified the set of characteristics and needs common to many IoT scenarios and made them available in one single reusable platform, offers a thorough overview of the platform’s architecture and related technologies (Akka, Kafka, Spark, Hadoop, etc.), demonstrates how Docker and CoreOS made the on-premises deployment easy, and reviews the generic analytics layer that uses machine learning to provide meaningful insights in different use cases and business domains. Moty concludes by discussing how Intel is using this platform to address problems that are not classical IoT use cases but can benefit from real-time analytics to achieve proactivity and operational excellence.
Moty Fania is a principal engineer and the CTO of the Advanced Analytics Group at Intel, which delivers AI and big data solutions across Intel. Moty has rich experience in ML engineering, analytics, data warehousing, and decision-support solutions. He led the architecture work and development of various AI and big data initiatives such as IoT systems, predictive engines, online inference systems, and more.
Comments on this page are now closed.
©2016, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.