Integrated data analytics for consumer electronics using Hadoop and Spark MLlib

Description

Panasonic, one of the largest home electronics product manufactures in the world, has launched a number of IoT home appliances connected to the cloud (air conditioners, rice cookers, microwave ovens, etc.). Along with providing cloud services to these products, Panasonic collects operation and state logs from the connected home appliances.

Takayuki Nishikawa and Ei Yamaguhi offer an overview of Panasonic’s recently developed integrated data analytics platform, which uses Hadoop and Hive as core components with NTT DATA (one of the largest system integrators in Japan, which is widely contributing to Hadoop and Spark community with several committers) in order to analyze the ever-increasing log data from these IoT home appliances. The platform makes the complicated data analytics process simple and has achieved scalability for millions of households and yielded an over 10x improvement in processing time. The platform also analyzes and extracts valuable information from multiappliance log data using machine learning with Spark MLlib, enabling Panasonic to get more reliable knowledge about users’ lifestyles.

Takayuki and Ei discuss the technical know-how obtained from the use case, designing and building the platform architecture, migrating existing processes, and constructing an analysis model with Spark MLlib.