Presented By O’Reilly and Cloudera
Make Data Work
21–22 May 2018: Training
22–24 May 2018: Tutorials & Conference
London, UK

Elastic map matching using Cloudera Altus and Apache Spark

Dr.-Ing. Michael Nolting (Volkswagen Commercial Vehicles)
14:5515:35 Wednesday, 23 May 2018
Data engineering and architecture
Location: S11B Level: Beginner
Secondary topics:  Transportation and Logistics

Who is this presentation for?

  • Data engineers and those working with geodata

Prerequisite knowledge

  • Familiarity with the Cloudera ecosystem and Spark

What you'll learn

  • Learn how Volkswagen Commercial Vehicle uses map matching for geodata and Altus to scale load over hybrid clouds
  • Understand howt o create an elastic map matching system

Description

Map matching describes the problem of mapping GPS coordinates that were recorded through a device to a geographical reference system, such as a map. Map matching applications exist in almost every telematics use case and are therefore crucial to car manufacturers, such as Volkswagen.

Volkswagen Commercial Vehicle’s Telematics Division offers analytical services to commercial customers, based on map-matched position data. These services include route optimization, smart service propositioning, and a driver’s logbook. Throughout a day the position data of a vehicle (e.g., the new Volkswagen Crafter) is collected into a central data store. A nightly running Spark batch job picks up the data from the central data store and applies the map matching operation to every vehicle’s position. The map-matched positions are then written back to the central data store and from there fed into downstream processing systems.

The Spark-based map matching application is submitted as an Altus job to an on-demand Cloudera cluster. The on-demand cluster gets automatically provisioned through Altus prior to submitting the Spark job and is released right after the map matching job has finished. This way Altus enables elastics resource provisioning and allows for acquiring compute resources only for as long as they are really needed, resulting in major savings. As an additional benefit, Altus allows for the compute cluster’s size growing in accordance to the amount of data that needs to be map-matched.

Michael Nolting details the architecture behind Volkswagen Commercial Vehicle’s Altus-based map matching application and leads a live demo featuring a map matching job in Altus.

Photo of Dr.-Ing. Michael Nolting

Dr.-Ing. Michael Nolting

Volkswagen Commercial Vehicles

Michael Nolting is a data scientist for Volkswagen commercial vehicles. Michael has worked in a variety of research fields at Volkswagen AG, including adapting big data technologies and machine learning algorithms to the automotive context. Previously, he was head of a big data analytics team at Sevenval Technologies. Michael holds a Dipl.-Ing. degree in electrical engineering and an MSc degree in computer science, both from the Technical University of Brunswick in Germany, and a PhD in computer science.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)