Presented By O'Reilly and Cloudera
Make Data Work
March 13–14, 2017: Training
March 14–16, 2017: Tutorials & Conference
San Jose, CA

The common anomaly detection platform at Microsoft

Tony Xing (Microsoft)
11:00am11:40am Thursday, March 16, 2017
Data engineering and architecture, Real-time applications
Location: LL20 A Level: Intermediate
Secondary topics:  Data Platform
Average rating: ***..
(3.00, 2 ratings)

Who is this presentation for?

  • Data engineers, data scientists, and data product managers

Prerequisite knowledge

  • A basic understanding of cloud services and predication-related ML algorithms

What you'll learn

  • Learn the engineering architecture, method, and algorithms applied in the common anomaly detection platform within Microsoft
  • Understand the problems the platform addresses and future work needed

Description

Microsoft’s Application and Service group had two systems doing time series anomaly detection for various data scenarios, which were serving teams across Bing Search, Ads, Office 365, and Skype. During the operation of those two systems, the team identified several customer pain points:

  • There are various time series signal types, and it is hard to have one algorithm cover everything with acceptable false positives.
  • Customers might not want to onboard specific data ingestion systems used by prior AD systems.
  • Customers want to have multidimensional anomaly detection (e.g., having dimensions like country, language, and devices), which is computational expensive.

Tony Xing offers an overview of Microsoft’s common anomaly detection platform, an API service built internally to provide product teams the flexibility to plug in any anomaly detection algorithms to fit their own signal types. This platform:

  • Handles multidimensional time series, so the anomalies within the dimensions and combinations of dimensions can be detected.
  • Is an independent API-based service, so customers can easily add AD into their own product experience.
  • Has a framework to easily plug in different learning algorithms to handle various signal types, so customers can pick what detection engine is best for them.
  • Operates in near real time.
  • Is a linear scalable service.
Photo of Tony Xing

Tony Xing

Microsoft

Tony Xing is a senior product manager on the Shared Data team within Microsoft’s Application and Service group. Previously, he was a senior product manager on the Skype data team within Microsoft’s Application and Service group. Tony recently gave a talk at Strata + Hadoop World in Beijing.

Comments on this page are now closed.

Comments

Picture of Tony Xing
Tony Xing | SENIOR PRODUCT MANAGER
03/16/2017 5:27am PDT

any discussion you can email to tonyxin@microsoft.com or find me on LinkedIn