Build resilient systems at scale
28–30 October 2015 • Amsterdam, The Netherlands

Analytics is the new monitoring: Anomaly detection applied to web performance

Bart De Vylder (CoScale)
14:40–15:20 Thursday, 29/10/2015
Location: G109
Average rating: ****.
(4.29, 14 ratings)
Slides:   1-PDF 

With the overload of metrics from your users, applications, and systems, keeping track of performance can be a daunting task. In this session we discuss how to apply data science techniques to technical and business metrics, in order to understand performance anomalies and their impact on your business.

We will discuss the following topics in more detail:

  • Anomaly detection in the context of web performance monitoring: use cases and challenges
  • Our experience with different machine learning techniques, and some practical examples
  • How to use context and knowledge of the system to your advantage vs. using generic mathematical models

This session is sponsored by CoScale

Photo of Bart De Vylder

Bart De Vylder

CoScale

Bart De Vylder is a data scientist at CoScale. Previously, Bart was active in software engineering and architecture, with a focus on distributed systems. His interests lie in machine learning and building reliable, scalable data processing systems. Bart holds a PhD in artificial intelligence from the Free University of Brussels.

Comments on this page are now closed.

Comments

Picture of Bart De Vylder
Bart De Vylder
29/10/2015 18:34 CET

Hi Kaupo,

That’s an interesting question. We have actually played with fourier analysis initially, but didn’t proceed along that road. At least for discovering the major frequency spikes we figured it’s better to just built in ‘domain knowlegde’ that 24h and to a lesser extent 7 days are the biggest ones. Filtering out the 24h cycle in the frequency domain, converting back to time domain as you suggest might be an interesting approach and I might try it out. At this point I can think of one issue that could arise in that you might introduce artifacts in the time-domain by such a filtering step causing anomalies at timestamps which were normal before.

Thanks for the remark,
Bart

Kaupo Arulo
29/10/2015 16:25 CET

Have you considered to convert your time series data into frequency domain? Then many type of digital filtering can be applied (filtering out 24h fluctuations etc.). When converting it back to time series then you probably do not need to look the 7*24h or 24h old data as a “baseline” to filter out noises and maybe higher sensibility can be achieved?