RT-Giraph: Online Graph Mining Simplified

Georgos Siganos (Qatar Computing Research Institute)
Hadoop Platform
Location: 114
Average rating: ***..
(3.00, 1 rating)

Several data mining applications depend on timely analysis of large, highly dynamic graphs. Existing graph mining architectures are, however, designed for offline computations making them unsuitable for dynamic graphs. At the same time, devising custom algorithms for dynamic graphs is a notoriously challenging algorithmic task, a fundamental obstacle in deploying online graph mining applications.

Real-Time (RT) Giraph aims to bridge the gap between offline and online graph mining with an architecture that offers the programming simplicity of batch processing while supporting efficient continuous processing of dynamic graphs. Central to RT-Giraph is a novel online graph mining engine that builds on the principles of Implicit Incremental Computation (IIC). The IIC engine computes graph algorithms using the Pregel model, and automatically updates the output through incremental computation as the graph changes.

In this presentation, we discuss the challenges in supporting incremental computation for graph algorithms and present the design and implementation of RT-Giraph, an Apache Giraph extension. RT-Giraph is an open-source project, part of the Grafos.ml effort to create tools for large-scale Machine Learning and Graph Analytics.

Georgos Siganos

Qatar Computing Research Institute

Georgos Siganos is a Senior Scientist at Qatar Computing Research Institute working on next generation Graph Mining Architectures and Big Data Systems. He is also the lead of the Grafos.ml open-source project. Previous to this, he was a Research Scientist at Telefonica Research focusing on Big Data and Peer to Peer Systems. He has authored more than 30 papers in journals and conferences. He received his Ph.D. from the University of California, Riverside.