Introducing a new anomaly detection algorithm (SR-CNN) inspired by computer vision
Who is this presentation for?Data engineers, data analyst, ML engineers, data analysts
Data driven companies need to monitor various metrics (for example, Page Views and Revenue) of their applications and services in real time. At Microsoft, we develop a time-series anomaly detection service which helps customers to monitor the time-series continuously and alert for potential incidents on time. In this talk, we introduce the pipeline and algorithm of our anomaly detection service, which is designed to be accurate, efficient and general. The pipeline consists of three major modules, including data ingestion, experimentation platform and online compute. To tackle the problem of time-series anomaly detection, we develop a novel algorithm based on Spectral Residual (SR) and Convolutional Neural Network (CNN). Our work is the first attempt to borrow the SR model from visual saliency detection domain to time-series anomaly detection. Moreover, we innovatively combine SR and CNN together to improve the performance of SR model. Our approach achieves superior experimental results compared with state-of-the-art baselines on both public datasets and Microsoft production data.
In the talk, we will walk through with audiences the following topics
1. Challenges with State of art methods
a) Lack of labels
2. System overview on how the combined novel algorithms + engineer system solves those challenges
a) Data Ingestion
b) Online Compute
c) Experimentation Platform
3. Applications within Microsoft
a) AIOps serving Bing, Office, Azure
b) Azure Cognitive Services
c) Selected case study
5. Result superior to SOTA time series anomaly detection algorithms
a) Data set
b) Experiment method
c) KPIs (precision, recall, F1) comparing with SOTA algorithms
6. Future work
Prerequisite knowledgeDeep learning, statistics methods, time series methods.
What you'll learn
Tony Xing is a Principal Product Manager in AI platform team within Microsoft’s Cloud + AI organization. Previously, he was a senior product manager on the AI/Data/Infra team and Skype data team within Microsoft’s Application and Service Group, where he worked on products for data ingestion, real-time data analytics, and the data quality platform.
Bixiong Xu is the principal dev manager on the AI Platform team at Microsoft Cloud + AI.
Congrui Huang is a senior data scientist at AI platform team of Microsoft Cloud + AI division.
I’m a program manager working on Big Data, Machine Learning and AI powered products for years. Recent focus is on time series anomaly detection and prediction to empower scenarios like AIOps related operation metrics detection, business metrics detection and prediction.
Leave a Comment or Question
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
For conference registration information and customer service
For more information on community discounts and trade opportunities with O’Reilly conferences
For information on exhibiting or sponsoring a conference
View a complete list of Strata Data Conference contacts