Skip to main content
Make Data Work
Oct 15–17, 2014 • New York, NY

Big Data Framework for Anomaly Detection & Root Cause Analysis on Streaming Time Series Data

Roy Singh (Guavus)
5:05pm–5:45pm Friday, 10/17/2014
Security
Location: 1 E10/1 E11
Tags: finance
Average rating: **...
(2.40, 5 ratings)

Data that is continuously being generated from machines, sensors from Internet of Things, mobile devices, network data traffic, and application logs contains valuable information into your business operations. By fusing, analyzing, and correlating these various data sets together in real-time, enterprises are able to have end-to-end visibility across their business to identify and detect situations relating to operational inefficiencies, opportunities to improve profitability, and reduce security threats.

Timely detection of outliers – anomalies and performing causality analysis for a close loop corrective actions as well as preventing future situations is very important. Some important use cases are failure detection nodes, dynamic load balancing of SDN, intrusion detection etc. However, the problem becomes complicated due to variety of factors influencing ability to detect and its computational intensity and finally a selection of algorithm(s). The factors include a number of dimensions, including hierarchical relations between the dimensions, correlation between influence of dimensions as well as data streams, number of observable KPIs, impact of seasonality on data, changes in the data due to trends, etc. These variations require different class of algorithms for detection.

In this session Guavus’ Chief Technology Officer, Roy Singh, will present a framework using an operational intelligence platform based on Apache Spark, for providing a pipeline for anomaly detection, causality analysis, anomaly prediction, and actionable alerts. For the illustration purpose, he will cover two approaches for anomaly detection, including a statistical approach over seasonal time series data and a subspace based anomaly detection over high dimensional multivariate use cases. Secondly he will cover an approach for causality analysis over anomalous events.

Photo of Roy Singh

Roy Singh

Guavus

Roy Singh is Chief Technology Officer at Guavus, the leading telecommunications data analytics solutions provider. He has been involved in the enterprise technology and data analytics industries for over 20 years. He spent 5 years in leadership roles in Microsoft’s Enterprise Architecture team, and several years as a consultant focused on data analytics and enterprise architecture in the government, retail, transportation and telecommunications industries. He also has previous experience as an entrepreneur in the financial technology industry, and began his career with 8 years at the Boston Consulting Group, where he worked on strategy, technology and process issues at a range of leading global enterprises. Roy holds an MBA from INSEAD, France, and graduated with a BA in Mathematics from Merton College, Oxford University.