Map matching describes the problem of mapping GPS coordinates that were recorded through a device to a geographical reference system, such as a map. Map-matching applications exist in almost every telematics use case and are therefore crucial to car manufacturers, such as Volkswagen.
Volkswagen Commercial Vehicle’s Telematics Division offers analytical services to commercial customers, based on map-matched position data. These services include route optimization, smart service propositioning, and a driver’s logbook. Throughout a day the position data of a vehicle (e.g., the new Volkswagen Crafter) is collected into a central data store. A nightly running Spark batch job picks up the data from the central data store and applies the map-matching operation to every vehicle’s position. The map-matched positions are then written back to the central data store and from there fed into downstream processing systems.
The Spark-based map-matching application is submitted as an Altus job to an on-demand Cloudera cluster. The on-demand cluster gets automatically provisioned through Altus prior to submitting the Spark job and is released right after the map matching job has finished. This way Altus enables elastics resource provisioning and allows for acquiring compute resources only for as long as they are really needed, resulting in major savings. As an additional benefit, Altus allows for the compute cluster’s size growing in accordance to the amount of data that needs to be map-matched.
Timo Graen and Robert Neumann detail the architecture behind Volkswagen Commercial Vehicle’s Altus-based map-matching application and lead a live demo featuring a map matching job in Altus.
Robert Neumann is founder and CEO of Ultra Tendency. He has more than a decade of experience designing and developing Hadoop-based applications and has been using Spark since v0.9.
©2018, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org