Introducing a new anomaly detection algorithm (SR-CNN) inspired by computer vision

Description

Data-driven companies need to monitor various metrics (e.g., page views and revenue) of their applications and services in real time. Microsoft developed a time series anomaly-detection service that helps customers monitor the time series continuously and alert for potential incidents on time.

Tony Xing, Congrui Huang, Qiyang Li, and Wenyi Yang introduce the pipeline and algorithm of Microsoft’s anomaly-detection service, which is designed to be accurate, efficient, and general. The pipeline consists of three major modules—data ingestion, experimentation platform, and online compute. To tackle the problem of time series anomaly detection, Microsoft developed a novel algorithm based on SR and CNN, which is the first attempt to borrow the SR model from the visual saliency-detection domain and apply it to time series anomaly detection. Moreover, SR and CNN are innovatively combined to improve the performance of the SR model, achieving superior experimental results compared with state-of-the-art baselines on both public datasets and Microsoft production data.

Along the way, you’ll discover some of the challenges of using state-of-the-art methods, such as lack of labels, generalization, and efficiency. Tony, Congrui, Qiyang, and Wenyi explain how Microsoft’s combined algorithms and system solves these challenges with data ingestion, online compute, and experimentation platform. You’ll get a sneak peak at some of these applications within Microsoft, including AIOps serving Bing, Office, and Azure; Azure Cognitive Services; and a selected case study. You’ll also see details of the methodology of the experiment and the superior results it provided as compared to state-of-the-art time series anomaly-detection algorithms, such as the dataset and experiment method (precision, recall, F1), and they’ll propose some future work.