Skip to main content

Driving the Future of Smart Cities - How to Beat the Traffic

Ian Huston (Pivotal), Alexander Kagoshima (Pivotal), Noelle Sio (Pivotal)
Machine Data
Mission City M
Average rating: ****.
(4.00, 1 rating)
Slides:   external link

As traffic volumes in cities around the world are constantly growing we are faced with the challenge to track and control car movements in a more detailed and intelligent way to beat the traffic. Real-time information on traffic including automotive sensors and crowd-sourced data feeds are an interesting new source of data. However, to utilize this data to its full extent and turn it into valuable information, intelligent methods for analyzing and predicting traffic are needed.

Pivotal’s Data Science Team has developed several innovative methods to analyze this traffic flow information harvested from real-time and in-car data sources including GPS. These methods by themselves are highly useful for predicting future traffic conditions and dissecting traffic data. We will describe how we created these algorithms and show different interesting results from their application. This example demonstrates how deeper insights into a problem can be found by combining different machine learning methods.

The methods developed by our team enable more intelligent routing systems through a more detailed velocity prediction based on a number of influencing factors. This is highly valuable for planning routes to far-away destinations and also useful on inner-city routes where traffic can be influenced by a lot of different factors. However, we recently tapped another valuable source of data that could enrich traffic prediction models even further.

Local transport authorities already make a lot of traffic and travel disruption freely available. These reports form the basis of traffic updates across a wide range of media. Currently however the reports are limited to acknowledging the start of a disruption, and then providing updates as the situation develops. In the smart city of the future these disruption reports will also predict the duration and severity of the disruptions, enabling route guidance systems to make better decisions.

We will also demonstrate a traffic disruption model that can predict the duration of recently begun incidents, learning the distinct traffic and disruption patterns of a major global city. The disruption prediction model incorporates historical traffic count data, previous incident reports and local weather conditions and uses an interesting variety of machine learning methods running on a massively parallel analytics database system.

We will conclude by outlining how the crowd-sourced real-time data could be matched to traffic disruption and open government data, to push the envelope in traffic analysis and prediction even further.

Photo of Ian Huston

Ian Huston

Data Scientist, Pivotal

Ian is a data scientist for Pivotal and works on a wide range of customer projects from fraud detection to transport and logistics.

Ian has a background in numerical analysis and simulation and his expertise includes high performance computing for scientific applications, perturbative analysis of large systems of differential equations and the differential geometry underlying relativistic physics.

He completed a PhD in theoretical cosmology at Queen Mary, University of London and received a MSc from Imperial College London in theoretical physics. Ian’s work has been published in leading international physics journals and he has released the Python numerical package used in his research to the community.

Alexander Kagoshima

Data Scientist, Pivotal

Alexander Kagoshima received a M.S. in Economics and Engineering from TU Berlin in 2012. In graduate school his focus was on machine learning and statistics. In his bachelor thesis he worked on applying Gaussian Processes to currency exchange rates. For his master thesis, Alex developed and evaluated a change-point detection algorithm that operates on wind data, to enable a new kind of intelligent wind-turbine control systems.

He gained practical experience in the application of machine learning methods as a working student at Volkswagen, his task was to analyze data of a test fleet of fuel-cell cars. Since December 2012 he works as a Data Scientist at Greenplum (now Pivotal) as the first Data Scientist in the EMEA team. In his spare time, he tries to find new ways to analyze soccer games through statistics.

Photo of Noelle Sio

Noelle Sio

Principal Data Scientist, Pivotal

Noelle Sio has a background in mathematics, statistics, and data mining with an emphasis on digital media. She is currently a Senior Data Scientist at Pivotal. Her work has mainly focused on helping companies extend their analytical capabilities by exploring and modeling digital data; from enabling a digital media agency to hypertarget their online campaigns to discovering new insights to online conversion drivers for a large retail bank. Previously, she worked as a researcher at eHarmony and Fox Interactive Media, where she leveraged massive datasets up to the petabyte level for marketing optimization, fraud detection, and ad monetization products. Noelle holds an A.B. From Washington University in St. Louis in Applied Mathematics and Physical Anthropology and a M.S. in Applied Mathematics from Cal Poly Pomona.