Presented By O'Reilly and Cloudera
Make Data Work
March 13–14, 2017: Training
March 14–16, 2017: Tutorials & Conference
San Jose, CA

Building reliable real-time services with Apache DistributedLog

Sijie Guo (StreamNative)
5:10pm5:50pm Wednesday, March 15, 2017
Secondary topics:  Media, Streaming
Average rating: **...
(2.00, 2 ratings)

What you'll learn

  • Learn how Twitter uses DistributedLog as its real-time data foundation in production


Apache DistributedLog (incubating) is a low-latency, high-throughput replicated log service. Sijie Guo shares how Twitter has used DistributedLog as the real-time data foundation in production for years, supporting services like distributed databases, pub-sub messaging, and real-time stream computing and delivering more than 1.5 trillion (17 PB) events per day.

Topics include:

  • Overview of Apache DistributedLog
  • How Twitter uses DistributedLog to build strong consistency in its databases
  • How Twitter uses DistributedLog for data replication across regions
  • How Twitter uses DistributedLog for pub-sub and real-time stream computing
  • Lessons learned from production
Photo of Sijie Guo

Sijie Guo


Sijie Guo is the founder and CEO of StreamNative, a data infrastructure startup offering a cloud native event streaming platform based on Apache Pulsar for enterprises. Previously, he was the tech lead for the Messaging Group at Twitter and worked on push notification infrastructure at Yahoo. He’s also the VP of Apache BookKeeper and PMC Member of Apache Pulsar.