Sep 9–12, 2019

Long-term real-time network traffic flow prediction using LSTM recurrent neural network

Wei Cai (Cox Communications)
2:35pm3:15pm Wednesday, September 11, 2019
Location: 230 C

Who is this presentation for?

  • Data scientists and machine learning modelers




Real-time traffic volume prediction plays a vital role in proactive network management, and many forecasting models have been proposed to address this issue. However, most of them suffer from the inability to fully use the rich information in traffic data to generate efficient and accurate traffic predictions for a longer term (i.e., seven-day predictions at a five-minute interval).

Wei Cai explores predicting multistep, real-time traffic volume using two types of long short-term memory (LSTM) networks: many-to-one LSTM and many-to-many LSTM by creating a flexible ensemble forecasting system that combines numbers of neural network and predictions out of interpolation. Considering the large number of data points in the current dataset, in order to address one of the typical concerns with recurrent neural network (RNN) models with respect to a longer training time, it was necessary to sample on the original dataset by each 12 data points (about an hour), and training was applied on the sampled data. Then 11 data points using interpolation between each pair of two sampled data points was done to effectively evaluate the model performance, with similar methods applied to generate a longer term of predictions at a five-minute time interval. Experimental results demonstrate that the proposed approach can effectively deal with the changing traffic pattern and show good performance in generating multistep predictions.

Based on experimental comparisons, many-to-many appears to be more appropriate for sequence predictions where multiple input time steps are required in order to predict a sequence of output time steps.

Prerequisite knowledge

  • Experience programming with Python
  • General knowledge of machine learning

What you'll learn

  • Understand how LSTM deep learning models can generate predictions for a little longer term
  • Learn about big data machine learners
Photo of Wei Cai

Wei Cai

Cox Communications

Wei Cai is a data scientist with Cox Communications, where she’s transitioned from building statistical models and doing analysis using R and SAS to developing machine learning models and processing streaming telemetry data using Python, Scala, and Java. She successfully developed both statistical models and deep learning models for the company to build a 10-year capacity planning budget, and she’s working on building models to compute to maximize network availability. She was born in China, earned her master’s degrees in actuarial science and business analysis from Georgia State University, and she’s seeking a master’s degree with a concentration in computing and analysis from the Georgia Institute of Technology. Her major areas of research interest are building deep learning models to help proactive network management.

  • Intel AI
  • O'Reilly
  • Amazon Web Services
  • IBM Watson
  • Dataiku
  • Dell Technologies
  • Intuit
  • Gamalon
  • Hewlett Packard Enterprise
  • MapR Technologies
  • Sisu Data
  • Intuit

Contact us

For conference registration information and customer service

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

For media/analyst press inquires