Presented By O'Reilly and Cloudera
Make Data Work
March 13–14, 2017: Training
March 14–16, 2017: Tutorials & Conference
San Jose, CA

Building reliable real-time services with Apache DistributedLog

Sijie Guo (ASF)
5:10pm5:50pm Wednesday, March 15, 2017
Secondary topics:  Media, Streaming
Average rating: **...
(2.00, 2 ratings)

What you'll learn

  • Learn how Twitter uses DistributedLog as its real-time data foundation in production

Description

Apache DistributedLog (incubating) is a low-latency, high-throughput replicated log service. Sijie Guo shares how Twitter has used DistributedLog as the real-time data foundation in production for years, supporting services like distributed databases, pub-sub messaging, and real-time stream computing and delivering more than 1.5 trillion (17 PB) events per day.

Topics include:

  • Overview of Apache DistributedLog
  • How Twitter uses DistributedLog to build strong consistency in its databases
  • How Twitter uses DistributedLog for data replication across regions
  • How Twitter uses DistributedLog for pub-sub and real-time stream computing
  • Lessons learned from production
Photo of Sijie Guo

Sijie Guo

ASF

Sijie Guo is the PMC chair of Apache BookKeeper and the PMC member of Apache Pulsar. He worked at Twitter before and led the messaging team. Prior to Twitter, he worked on Yahoo! push notification infrastructure.