Recurrent neural networks for time series forecasting

Description

Time series forecasting is one of the most important topics in data science. Almost every business needs to predict the future in order to make better decisions and allocate resources more effectively. Examples of time series forecasting use cases include financial forecasting, product sales forecasting, web traffic forecasting, energy demand forecasting for buildings and data centers, and many more. However, most existing forecasting solutions use traditional time series and machine learning models. For complex forecasting problems, data scientists need to know how to leverage advanced techniques to generate more accurate forecasts.

Recurrent neural networks (RNNs), as opposed to feedforward neural networks, are designed for processing sequential data. Recently, RNNs have achieved a lot of success and are frequently used in text, speech, and video analysis but are less used for time series forecasting.

Join Yijing Chen, Dmitry Pechyoni, Angus Taylor, and Vanja Paunic to explore the basic concepts of building RNN models and learn how and when to apply them to time series forecasting. Yijing, Dmitry, Angus, and Vanja start with a quick overview of time series forecasting and traditional time series models before sharing a comprehensive introduction covering everything from feedforward neural networks to recurrent neural networks. You’ll then discover how RNN models are trained and dive into different RNN architectures, such as LSTM (long short-term memory) and GRU (gated recurrent unit).
 
Yijing, Dmitry, Angus, and Vanja conclude by examining how and when to use RNNs for time series forecasting. You’ll learn how to prepare time series data and how to use Keras to implement RNN models. Along the way, they share best practices for building state-of-the-art time series forecasting models using RNNs along with simple examples and successful applications of RNNs in Kaggle competitions.
 
Outline:

Intro to time series forecasting

• What is time series forecasting, and why it is important?
• General description of time series data and time series analytics
• General data preparation for time series forecasting
• Time series models
   
  Introduction to RNN architectures and training

• Introduction to feedforward neural networks
  • Basic concepts (neurons, layers, weights, bias, and activation functions)
  • Cost function
  • Training using stochastic gradient descent and minibatches
  • Backpropagation
  • Early stopping
• Introduction to recurrent neural networks (RNNs)
• Backpropagation through time (BPTT)
• Vanishing gradient and exploding gradient
• Comparison of different RNN units
  • GRU
  • LSTM
• Stateful and stateless training of recurrent neural networks
   
  How and when to apply RNNs to time series forecasting

• Software packages for training RNN (Keras, TensorFlow, CNTK, etc.)
• Data preprocessing
• How to tune RNNs
  • Hyperparameters
  • Network architecture (deep RNN, encoder-decoder architecture)
  • Initialization
  • Regularization
  • Dropout and zoneout
• Recursive and multioutput strategies for multistep forecasting
• Probabilistic forecasting with RNNs
• Practical examples
  • Walkthrough of simple RNN models with energy demand forecasting use case
  • Walkthrough of successful RNN models that achieved top scores in Kaggle competitions (web traffic forecasting and grocery sales forecasting)
  • Comparison of RNN with traditional time series (e.g., ARIMA and ETS) and machine learning (e.g., random forest and boosted decision trees) methods