Presented By O’Reilly and Cloudera
Make Data Work
21–22 May 2018: Training
22–24 May 2018: Tutorials & Conference
London, UK

Elastic Map Matching Using Cloudera Altus and Apache Spark

Dr.-Ing. Michael Nolting (Volkswagen Commercial Vehicles)
14:5515:35 Wednesday, 23 May 2018
Data engineering and architecture
Location: S11B Level: Beginner

Who is this presentation for?

Data engineers; Geodata guys

Prerequisite knowledge

Cloudera ecosystem Spark

What you'll learn

- What is map matching for geodata and why it is needed - Howto create an elastic map matching system - Howto use Altus to scale load over hybrid clouds

Description

Map matching describes the problem of mapping GPS coordinates that were recorded through a device to a geographical reference system, such as a map. Map matching applications exist in almost every telematics use case and are therefore crucial to car manufacturers, such as Volkswagen.

Volkswagen Commercial Vehicle’s telematics division offers analytical services to their commercial customers. These analytical services are based on map-matched position data and include route optimization, smart service propositioning, as well as a driver’s logbook.

Throughout a day the position data of a vehicle (e.g., the new Volkswagen Crafter) is collected into a central data store. A nightly running Spark batch job picks up the data from the central data store and applies the map matching operation to every vehicle’s position. The map-matched positions are then written back to the central data store and from there fed into downstream processing systems.

The Spark-based map matching application is submitted as an Altus job to an on-demand Cloudera cluster. The on-demand cluster gets automatically provisioned through Altus prior to submitting the Spark job and is released right after the map matching job has finished. This way Altus enables elastics resource provisioning and allows for acquiring compute resources only for as long as they are really needed resulting into major cost saving advantages. As an additional benefit, Altus allows for the compute cluster’s size growing in accordance to the amount of data that needs to be map-matched.

This talk details the architecture behind Volkswagen Commercial Vehicle’s Altus-based map matching application. It closes with a live demo featuring the map matching job in Altus.

Photo of Dr.-Ing. Michael Nolting

Dr.-Ing. Michael Nolting

Volkswagen Commercial Vehicles

Michael Nolting is a data scientist for Volkswagen commercial vehicles. Michael has worked in a variety of research fields at Volkswagen AG, including adapting big data technologies and machine learning algorithms to the automotive context. Previously, he was head of a big data analytics team at Sevenval Technologies. Michael holds a Dipl.-Ing. degree in electrical engineering and an MSc degree in computer science, both from the Technical University of Brunswick in Germany, and a PhD in computer science.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)