Swisscom, the leading mobile service provider in Switzerland, also provides data-driven intelligence through the analysis of the data created by its mobile network. Its Mobility Insights team works to help civil administrators, tourism and marketing professionals, and many others understand the flow of people through their locations of interest. François Garillot outlines the platform, tooling, and choices that help achieve this service and some challenges the team has faced, before exploring in depth the task of understanding the speeds of populations through a path of interest.
François offers an overview of the design of Swisscom’s big data infrastructure, which features Scala, Kafka, and Spark as pivotal tools, focusing on the multiple components that allow unified and reliable access to the high-throughput data flowing from the telecommunication network and lead to a single platform that lets engineers answer heterogeneous questions quickly. Along the way, François explains the technical challenges of moving into real-time analysis and fast data, a key feature of the speed measurement task.
François also proposes possible solutions to technical challenges, such as selecting interesting datapoints out of a millions coming in every second, sessionizing when no batch interval seems to make clear sense, and the importance of checking ground truth, and explains how privacy protection—crucial to Swisscom and the Mobility Insights team—is constitutive of both the data filtering and the questions the team chooses to tackle.
François Garillot is a data scientist at Swisscom, where he works on curating and understanding telecommunications data through big data tools. Previously, François worked on Apache Spark Streaming’s reliability at Lightbend (formerly Typesafe). His interests include machine learning—especially online models, approximation and hashing techniques, control theory, and unsupervised time series analysis—skiing, sailing, and hunting for good cheese.
Comments on this page are now closed.
©2016, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.