Presented By O'Reilly and Cloudera
Make Data Work
March 13–14, 2017: Training
March 14–16, 2017: Tutorials & Conference
San Jose, CA

Building reliable real-time services with Apache DistributedLog

Sijie Guo (Apache Software Foundation)
5:10pm5:50pm Wednesday, March 15, 2017
Secondary topics:  Media, Streaming
Average rating: **...
(2.00, 2 ratings)

What you'll learn

  • Learn how Twitter uses DistributedLog as its real-time data foundation in production


Apache DistributedLog (incubating) is a low-latency, high-throughput replicated log service. Sijie Guo shares how Twitter has used DistributedLog as the real-time data foundation in production for years, supporting services like distributed databases, pub-sub messaging, and real-time stream computing and delivering more than 1.5 trillion (17 PB) events per day.

Topics include:

  • Overview of Apache DistributedLog
  • How Twitter uses DistributedLog to build strong consistency in its databases
  • How Twitter uses DistributedLog for data replication across regions
  • How Twitter uses DistributedLog for pub-sub and real-time stream computing
  • Lessons learned from production
Photo of Sijie Guo

Sijie Guo

Apache Software Foundation

Sijie Guo is the PMC chair of Apache BookKeeper and the PMC member of Apache Pulsar at the Apache Software Foundation. Previously, he led the messaging team at Twitter and worked on push notification infrastructure at Yahoo.