Presented By O'Reilly and Cloudera
Make Data Work
31 May–1 June 2016: Training
1 June–3 June 2016: Conference
London, UK

Anomaly detection in telecom with Spark

Ted Dunning (MapR, now part of HPE)
14:55–15:35 Thursday, 2/06/2016
Spark & beyond
Location: Capital Suite 13 Level: Intermediate
Average rating: ****.
(4.38, 13 ratings)

Prerequisite knowledge

Attendees should have a general knowledge of programming.


Telecom operators need to find operational anomalies in their networks very quickly, a need shared by many other industries. Spark plus a streaming architecture can solve these problems very nicely. Ted Dunning presents a practical architecture as well as some detailed algorithms for detecting anomalies in event streams. These algorithms are simple and quite general and can be applied across a wide variety of situations.

All code for this talk will be open source and available on GitHub.

Photo of Ted Dunning

Ted Dunning

MapR, now part of HPE

Ted Dunning is the chief technology officer at MapR, an HPE company. He’s also a board member for the Apache Software Foundation, a PMC member, and committer on a number of projects. Ted has years of experience with machine learning and other big data solutions across a range of sectors. He’s contributed to clustering, classification, and matrix decomposition algorithms in Mahout and to the new Mahout Math library and designed the t-digest algorithm used in several open source projects and by a variety of companies. Previously, Ted was chief architect behind the MusicMatch (now Yahoo Music) and Veoh recommendation systems and built fraud-detection systems for ID Analytics (LifeLock). Ted has coauthored a number of books on big data topics, including several published by O’Reilly related to machine learning, and has 24 issued patents to date plus a dozen pending. He holds a PhD in computing science from the University of Sheffield. When he’s not doing data science, he plays guitar and mandolin. He also bought the beer at the first Hadoop user group meeting.