Sep 23–26, 2019

Introducing a new anomaly detection algorithm (SR-CNN) inspired by computer vision

Tony Xing (Microsoft), Congrui Huang (Microsoft), Qiyang Li (Microsoft), Wenyi Yang (Microsoft)
2:55pm3:35pm Wednesday, September 25, 2019
Location: 1A 06/07
Average rating: ****.
(4.67, 3 ratings)

Who is this presentation for?

  • Data engineers, data analysts, and ML engineers

Level

Intermediate

Description

Data-driven companies need to monitor various metrics (e.g., page views and revenue) of their applications and services in real time. Microsoft developed a time series anomaly-detection service that helps customers monitor the time series continuously and alert for potential incidents on time.

Tony Xing, Congrui Huang, Qiyang Li, and Wenyi Yang introduce the pipeline and algorithm of Microsoft’s anomaly-detection service, which is designed to be accurate, efficient, and general. The pipeline consists of three major modules—data ingestion, experimentation platform, and online compute. To tackle the problem of time series anomaly detection, Microsoft developed a novel algorithm based on SR and CNN, which is the first attempt to borrow the SR model from the visual saliency-detection domain and apply it to time series anomaly detection. Moreover, SR and CNN are innovatively combined to improve the performance of the SR model, achieving superior experimental results compared with state-of-the-art baselines on both public datasets and Microsoft production data.

Along the way, you’ll discover some of the challenges of using state-of-the-art methods, such as lack of labels, generalization, and efficiency. Tony, Congrui, Qiyang, and Wenyi explain how Microsoft’s combined algorithms and system solves these challenges with data ingestion, online compute, and experimentation platform. You’ll get a sneak peak at some of these applications within Microsoft, including AIOps serving Bing, Office, and Azure; Azure Cognitive Services; and a selected case study. You’ll also see details of the methodology of the experiment and the superior results it provided as compared to state-of-the-art time series anomaly-detection algorithms, such as the dataset and experiment method (precision, recall, F1), and they’ll propose some future work.

Prerequisite knowledge

  • A basic understanding of deep learning, statistics methods, and time series methods

What you'll learn

  • Learn about the novel time series anomaly-detection algorithm inspired by the computer vision deep learning domain, which is far superior to state-of-the-art methods, and the application of the algorithm and system in cognitive services and AIOps in Microsoft
Photo of Tony Xing

Tony Xing

Microsoft

Tony Xing is a senior product manager on the AI, data, and infrastructure (AIDI) team within Microsoft’s AI and Research Organization. Previously, he was a senior product manager on the Skype data team within Microsoft’s Application and Service Group, where he worked on products for data ingestion, real-time data analytics, and the data quality platform.

Photo of Congrui  Huang

Congrui Huang

Microsoft

Congrui Huang is a senior data scientist on the AI platform team within the Cloud and AI Division at Microsoft.

Photo of Qiyang Li

Qiyang Li

Microsoft

Qiyang Li is a a program manager at Microsoft, having had worked on big data, machine learning and AI-powered products for years. His recent focus is on time series-anomaly detection and prediction to empower scenarios like AIOps-related operation metrics detection, business metrics detection, and prediction.

Wenyi Yang

Microsoft

Wenyi Yang is a software engineer on the AI platform team at Microsoft.

Comments on this page are now closed.

Comments

Emily Zhou | Senior Data Scientist
09/20/2019 12:40pm EDT

Hi,

I have two questions:

1. what use case was this algorithm built for?
2. could you touch upon what methodology you used to handle the unbalanced data, and why?

Thanks,
Emily

    Contact us

    confreg@oreilly.com

    For conference registration information and customer service

    partners@oreilly.com

    For more information on community discounts and trade opportunities with O’Reilly conferences

    strataconf@oreilly.com

    For information on exhibiting or sponsoring a conference

    pr@oreilly.com

    For media/analyst press inquires