The internet of things (IoT) is producing a tremendous amount of data from sensors on manufacturing equipment, vehicles, heavy machinery, wearables, smartphones, and more. This data has the power to dramatically improve the utility of these products, but the impact of the IoT seems to have fallen short of its promise.
A recent study by McKinsey & Company showed an oil rig with 30,000 sensors only had 1% of its data examined, proving that the best sensors and most innovative instrumentation are still only as good as the tools used to store and analyze that data. To handle the sheer volume of IoT data, architects often use a NoSQL-style database, such as Apache HBase or Apache Cassandra, which enables ingest and simple monitoring, but lacks the analytics needed for optimization and prediction. To bridge the gap to running analytics on IoT data, practitioners looked to deploy a lambda architecture, which pushed data from NoSQL storage into HDFS to enable analytics. However, this was a difficult process full of potential problems.
Michael Crutcher and Ryan Lippert explain why Apache Kudu, a relational storage layer for fast analytics on fast data, is the key to unlocking the value in IoT data. Apache Kudu provides a place where users can land all of their data in real time and then make it instantly available for analytic scans. By simplifying this architecture with open source software, Kudu enables organizations to build IoT models that can do machine learning and complex event processing, finally delivering on the promises of optimization and prediction that the IoT has been making for a decade.
Michael Crutcher is the director of product management at Cloudera, where he is responsible for the direction of Cloudera’s storage products, which include HDFS, HBase, and Parquet. He’s also responsible for managing strategic partnerships with storage vendors.
Ryan Lippert is a senior product marketing manager at Cloudera, where he is responsible for the company’s Operational Database offering and for marketing its storage products. Previously, Ryan served in a variety of roles at Cisco Systems. He holds an economics degree from the University of Guelph and an MBA from Stanford.
©2017, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org