Deep learning for time series data

Description

Ira Cohen shares a novel two-step approach for building more reliable prediction models by integrating anomalies in them. The first step uses anomaly detection algorithms to discover anomalies in a time series in the training data. In the second, multiple prediction models, including time series models and deep networks, are trained, enriching the training data with the information about the anomalies discovered in the first step.

Anomaly detection for individual time series is a necessary but insufficient step, due to the fact that anomaly detection over a set of live data streams may result in anomaly fatigue, thereby limiting effective decision making. One way to address the above is to carry out anomaly detection in a multidimensional space. However, this is typically very expensive computationally and hence not suitable for live data streams. Another approach is to carry out anomaly detection on individual data streams and then leverage correlation analysis to minimize false positives, which in turn helps in surfacing actionable insights faster.

Arun Kejariwal walks you through how to marry correlation analysis with anomaly detection, discusses how the topics are intertwined, and details the challenges you may encounter based on production data. Arjun also showcases how deep learning can be leveraged to learn nonlinear correlation, which in turn can be used to further contain the false positive rate of an anomaly detection system.