Always wanted to know where those colorful flying machines whooshing by overhead are going? Ever wondered how you could incorporate several technology domains into one nice (intermediate) project? Do you want to make your own antenna, receive signals from airplanes, and make them available in your Hadoop cluster for further data analyses? Always wanted to know what the busiest routes in your area are, or what the chances are a certain plane is going to a certain location? Predict airplane traffic/routes based on weather conditions?
Hellmar Becker and Jorn Eilander explore the real-time collection and predictive analytics of flight radar data with IoT devices, NiFi, HBase, Spark, and Zeppelin. Hellmar and Jorn use a set of Raspberry Pis in different locations and a self-built 1090 MHz antenna to collect airplane data over a large geographical area in a distributed way. The Raspberry Pis run MiNiFi to send data to a NiFi service, which feeds the main analytics Hadoop cluster in the cloud. Hellmar and Jorn explain how to use this data, using Spark and HBase, to predict airplane traffic and volume based on weather conditions and potentially predict an airplane’s destination.
Hellmar and Jorn then demonstrate one of their collection devices and walk you through building a rig of your own and using Hadoop to make use of this very nice readily available source of data. It’s a good way to start learning about several technology domains and see what you can do with NiFi, Hadoop, and a few dollars worth of hardware.
Hellmar Becker is a solutions engineer at Hortonworks, where he is helping spread the word about what you can do with data in the modern world. Hellmar has worked in a number of positions in big data analytics and digital analytics. Previously, he worked at ING Bank implementing the Datalake Foundation project (based on Hadoop) within client information management.
Jorn Eilander is a Hadoop DevOps Engineer at ING. Jorn has lots of experience working with Hadoop in high-risk impacted enterprise environments as a data ingestion expert and Hadoop system engineer. As an IoT and home automation enthusiast, he’s worked with several Raspberry Pi and Arduino platforms to gather data for his Hadoop cluster.
Comments on this page are now closed.
©2017, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com