Presented By O’Reilly and Intel AI
Put AI to work
8-9 Oct 2018: Training
9-11 Oct 2018: Tutorials & Conference
London, UK

Recurrent neural networks for time series forecasting

Yijing Chen (Microsoft), Dmitry Pechyoni (Microsoft), Angus Taylor (Microsoft), Vanja Paunic (Microsoft)
13:30–17:00 Tuesday, 9 October 2018
Models and Methods
Location: Buckingham Room - Palace Suite
Secondary topics:  Deep Learning models, Financial Services, Temporal data and time-series
Average rating: ***..
(3.67, 3 ratings)

Who is this presentation for?

  • Data scientists and engineers

Prerequisite knowledge

  • A basic understanding of machine learning concepts and models
  • No experience with time series or neural networks required

Materials or downloads needed in advance

  • A laptop with Microsoft Azure Notebooks and course code and data set up prior to the course (instructions on the course GitHub repository)

Microsoft Azure Notebooks setup:

  1. Go to https://notebooks.azure.com/ and click "Sign In" on the top right.
  2. Use your Microsoft account to sign in. If you don't have a personal Microsoft account, create one. (You should use your personal Microsoft account. If you use your organizational account, you will need to go through the login process by your organization.)
  3. Create a user ID.

Tutorial code and data setup:

  1. Log in to Azure Notebooks. Go to "Libraries" on the top left and click "+ New Library."
  2. In the pop-out window, select "From GitHub" tab, and in "GitHub repository" type "Azure/RNNForTimeSeriesForecasting." Then type in any name for "Library Name" and "Library ID." Once you have filled in all boxes, click "Import." Wait untill you see a list of files cloned from the GitHub repository to your library.
  3. To set up your notebook environment, click "Settings." In the pop-out window, select the "Environment" tab and click "+ Add." Choose "Select Operation" from the drop-down menu and select "Requirements.txt." For "Select Target File," choose "Requirements.txt." For "Select Python Version," choose "Python Version 3." Then click "Save."
  4. Go back to your library and click "0_data_setup.ipynb." You may get the message "Waiting for your container to finish being prepared…" It will take about two minutes for Azure Notebooks to set up your environment the first time you open a notebook in your library. Please wait until this message disappears before you move to the next step.
  5. Before you run the notebook, make sure you see "Python 3" kernel on the top right. If not, you can select "Kernel" and "Change kernel" to make changes.
  6. Run each cell in the notebook by clicking "Run." This notebook will download sample data to your environment and visualize the data. Wait until you are able to see all the visualizations. Now you are all set. (If you see errors return from the first code cell, it's very likely that the environment preparation is not finished yet. Please wait a few minutes. Then go to "Kernel" and choose "Restart and Clear Output" and rerun the cells.)

What you'll learn

  • Understand the basics of recurrent neural networks (RNN) and advanced RNN architectures, including LSTM and GRU
  • Learn when to use RNNs instead of traditional time series models in time series forecasting
  • Explore a number of techniques and tricks that are important for building successful RNN-based time series forecasting models
  • Get access to the source code of examples of how to train RNN models with Keras on Azure

Description

Time series forecasting is one of the most important topics in data science. Almost every business needs to predict the future in order to make better decisions and allocate resources more effectively. Examples of time series forecasting use cases include financial forecasting, product sales forecasting, web traffic forecasting, energy demand forecasting for buildings and data centers, and many more. However, most existing forecasting solutions use traditional time series and machine learning models. For complex forecasting problems, data scientists need to know how to leverage advanced techniques to generate more accurate forecasts.

Recurrent neural networks (RNNs), as opposed to feedforward neural networks, are designed for processing sequential data. Recently, RNNs have achieved a lot of success and are frequently used in text, speech, and video analysis but are less used for time series forecasting.

Join Yijing Chen, Dmitry Pechyoni, Angus Taylor, and Vanja Paunic to explore the basic concepts of building RNN models and learn how and when to apply them to time series forecasting. Yijing, Dmitry, Angus, and Vanja start with a quick overview of time series forecasting and traditional time series models before sharing a comprehensive introduction covering everything from feedforward neural networks to recurrent neural networks. You’ll then discover how RNN models are trained and dive into different RNN architectures, such as LSTM (long short-term memory) and GRU (gated recurrent unit).
 
Yijing, Dmitry, Angus, and Vanja conclude by examining how and when to use RNNs for time series forecasting. You’ll learn how to prepare time series data and how to use Keras to implement RNN models. Along the way, they share best practices for building state-of-the-art time series forecasting models using RNNs along with simple examples and successful applications of RNNs in Kaggle competitions.
 
Outline:

Intro to time series forecasting

  • What is time series forecasting, and why it is important?
  • General description of time series data and time series analytics
  • General data preparation for time series forecasting
  • Time series models
     
    Introduction to RNN architectures and training
  • Introduction to feedforward neural networks
    • Basic concepts (neurons, layers, weights, bias, and activation functions)
    • Cost function
    • Training using stochastic gradient descent and minibatches
    • Backpropagation
    • Early stopping
  • Introduction to recurrent neural networks (RNNs)
  • Backpropagation through time (BPTT)
  • Vanishing gradient and exploding gradient
  • Comparison of different RNN units
    • GRU
    • LSTM
  • Stateful and stateless training of recurrent neural networks
     
    How and when to apply RNNs to time series forecasting
  • Software packages for training RNN (Keras, TensorFlow, CNTK, etc.)
  • Data preprocessing
  • How to tune RNNs
    • Hyperparameters
    • Network architecture (deep RNN, encoder-decoder architecture)
    • Initialization
    • Regularization
    • Dropout and zoneout
  • Recursive and multioutput strategies for multistep forecasting
  • Probabilistic forecasting with RNNs
  • Practical examples
    • Walkthrough of simple RNN models with energy demand forecasting use case
    • Walkthrough of successful RNN models that achieved top scores in Kaggle competitions (web traffic forecasting and grocery sales forecasting)
    • Comparison of RNN with traditional time series (e.g., ARIMA and ETS) and machine learning (e.g., random forest and boosted decision trees) methods
Photo of Yijing Chen

Yijing Chen

Microsoft

Yijing Chen is a senior data scientist in the Cloud AI Group at Microsoft, where she works with external customers in areas such as energy demand forecast, user mobile behavioral analysis, retail demand forecast, energy theft detection, product pricing, and medical claim denial prediction as well as on other projects using various machine learning methods. Yijing holds an MA in statistics from Harvard University.

Photo of Dmitry Pechyoni

Dmitry Pechyoni

Microsoft

Dmitry Pechyoni is a senior data scientist in the Cloud AI Group at Microsoft, where he works on building end-to-end data science solutions in various domains, including retail, energy management, and predictive maintenance. Previously, he built machine learning models for display advertising Akamai and MediaMath. Dmitry holds a PhD in theoretical machine learning from the Technion – Israel Institute of Technology.

Photo of Angus Taylor

Angus Taylor

Microsoft

Angus Taylor is a data scientist in the Cloud AI Group at Microsoft, where he builds data science solutions for external customers in the retail, energy, engineering, and package distribution sectors. He holds an MSc in AI from the University of Edinburgh.

Photo of Vanja Paunic

Vanja Paunic

Microsoft

Vanja Paunic is a data scientist in the Algorithms and Data Science Group at Microsoft London. She works on building machine learning solutions with external companies utilizing Microsoft’s AI Cloud Platform. She holds a PhD in computer science with a focus on data mining in the biomedical domain from the University of Minnesota.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Comments

Picture of Yijing Chen
Yijing Chen | DATA SCIENTIST
25/09/2018 19:33 BST

Hi Olga, Walter, I believe you can contact the conference organizer confreg@oreilly.com for further questions about pricing.

Olga Kolchyna | AI/MACHINE LEARNING RESEARCHER
24/09/2018 12:51 BST

The same question as Walter asked. Who can I get in touch with to get a price for Tuesday session only? Thanks, Olga

Walter Hofmann |
25/08/2018 11:28 BST

I like to join only this session during the AI conference. Therefore, I like to ask whether you can provide a special price for it.
Regards,
Walter