Sep 23–26, 2019

Introducing a new anomaly detection algorithm (SR-CNN) inspired by Computer Vision

Tony Xing (Microsoft), Bixiong Xu (Microsoft), Congrui Huang (Microsoft), Qun Ying (Microsoft)
2:55pm3:35pm Wednesday, September 25, 2019
Location: 1A 06/07
Secondary topics:  Deep Learning, Temporal data and time-series analytics

Who is this presentation for?

Data engineers, data analyst, ML engineers, data analysts

Level

Intermediate

Prerequisite knowledge

Deep learning, statistics methods, time series methods.

What you'll learn

A novel time series anomaly detection algorithm inspired from computer vision deep learning domain, which is far superior than SOTA methods. The application of the algorithm and system in Cognitive Services and AIOps in Microsoft.

Description

Data driven companies need to monitor various metrics (for example, Page Views and Revenue) of their applications and services in real time. At Microsoft, we develop a time-series anomaly detection service which helps customers to monitor the time-series continuously and alert for potential incidents on time. In this talk, we introduce the pipeline and algorithm of our anomaly detection service, which is designed to be accurate, efficient and general. The pipeline consists of three major modules, including data ingestion, experimentation platform and online compute. To tackle the problem of time-series anomaly detection, we develop a novel algorithm based on Spectral Residual (SR) and Convolutional Neural Network (CNN). Our work is the first attempt to borrow the SR model from visual saliency detection domain to time-series anomaly detection. Moreover, we innovatively combine SR and CNN together to improve the performance of SR model. Our approach achieves superior experimental results compared with state-of-the-art baselines on both public datasets and Microsoft production data.

In the talk, we will walk through with audiences the following topics

1. Challenges with State of art methods
a) Lack of labels
b) Generalization
c) Efficiency

2. System overview on how the combined novel algorithms + engineer system solves those challenges
a) Data Ingestion
b) Online Compute
c) Experimentation Platform

3. Applications within Microsoft
a) AIOps serving Bing, Office, Azure
b) Azure Cognitive Services
c) Selected case study

4. Methodology
a) SR
b) SR-CNN

5. Result superior to SOTA time series anomaly detection algorithms
a) Data set
b) Experiment method
c) KPIs (precision, recall, F1) comparing with SOTA algorithms

6. Future work

Photo of Tony Xing

Tony Xing

Microsoft

Tony Xing is a Principal Product Manager in AI platform team within Microsoft’s Cloud + AI organization. Previously, he was a senior product manager on the AI/Data/Infra team and Skype data team within Microsoft’s Application and Service Group, where he worked on products for data ingestion, real-time data analytics, and the data quality platform.

Photo of Bixiong Xu

Bixiong Xu

Microsoft

Bixiong Xu is the principal dev manager on the AI Platform team at Microsoft Cloud + AI.

Photo of Congrui  Huang

Congrui Huang

Microsoft

Congrui Huang is a senior data scientist at AI platform team of Microsoft Cloud + AI division.

Photo of Qun Ying

Qun Ying

Microsoft

Qun Ying is a Sr. product manager @ AI Platform team of Microsoft Cloud + AI division

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

strataconf@oreilly.com

For information on exhibiting or sponsoring a conference

Contact list

View a complete list of Strata Data Conference contacts