Presented By O'Reilly and Cloudera
Make Data Work
March 13–14, 2017: Training
March 14–16, 2017: Tutorials & Conference
San Jose, CA

Building reliable real-time services with Apache DistributedLog

Sijie Guo (Streamlio)
5:10pm5:50pm Wednesday, March 15, 2017
Secondary topics:  Media, Streaming
Average rating: **...
(2.00, 2 ratings)

What you'll learn

  • Learn how Twitter uses DistributedLog as its real-time data foundation in production


Apache DistributedLog (incubating) is a low-latency, high-throughput replicated log service. Sijie Guo shares how Twitter has used DistributedLog as the real-time data foundation in production for years, supporting services like distributed databases, pub-sub messaging, and real-time stream computing and delivering more than 1.5 trillion (17 PB) events per day.

Topics include:

  • Overview of Apache DistributedLog
  • How Twitter uses DistributedLog to build strong consistency in its databases
  • How Twitter uses DistributedLog for data replication across regions
  • How Twitter uses DistributedLog for pub-sub and real-time stream computing
  • Lessons learned from production
Photo of Sijie Guo

Sijie Guo


Sijie Guo is the co-founder of Streamlio that focuses on building next generation real time data stack. Before Streamlio, he was the tech lead for messaging group at Twitter where he co-created Apache DistributedLog. He is also the PMC chair of Apache BookKeeper. Prior to Twitter, he worked on Yahoo! push notification infrastructure at Yahoo.