O'Reilly、Cloudera 主办
Make Data Work

现代流计算架构 (Modern streaming architectures)

此演讲使用中文 (This will be presented in Chinese)

Sijie Guo (StreamNative), Maosong Fu (Twitter)
13:30–17:00 Thursday, 2017-07-13
物联网&实时计算 (IoT & real-time)
地点: 多功能厅5A(Function Room 5A) 观众水平 (Level): Beginner

必要预备知识 (Prerequisite Knowledge)

Basic knowledge about stream computing systems, messaging/stream storage systems, and distributed systems

需要提前准备的资料和下载 (Materials or downloads needed in advance)

A laptop (Linux or macOS) with DistributedLog and Heron installed

您将学到什么 (What you'll learn)

Explore modern streaming architectures, Twitter's real-time stack, and lessons learned running a real-time stack at Twitter's scale

描述 (Description)


- 流计算的简介,以及一些典型应用
- 流计算架构是什么?
- 不同类型的流计算架构和他们的优缺点
- 详细讨论为实时数据存储设计的Apache DistributedLog,以及它在现代实时数据技术栈里的使用场景
- 详细讨论流计算引擎Heron,以及它在现代实时数据技术栈里的使用场景
- Twitter在使用Apache DistributedLog和Heron来搭建实时数据技术栈时获得的经验教训。

The move to streaming architectures from batch processing is a revolution in how companies use data. Twitter, for instance, is all about real time and has been at the forefront of investing and developing a real-time stack of several years, and recently, there has been a lot of interest in real time from enterprises as well. But what is the state of the art for a real-time data stack? Sijie Guo and Maosong Fu explore the typical challenges in a modern real-time data stack and explain how the modern technology will impact streaming architecture and applications in the future.

Topics include:

  • An introduction to streaming and some sample applications
  • Streaming architecture overview
  • Different types of streaming architectures and their pros and cons
  • An in-depth discussion of Apache DistributedLog for storage
  • An in-depth discussion of Heron
  • Lessons learned from building a real-time stack using Apache DistributedLog and Heron at Twitter
Photo of Sijie Guo

Sijie Guo


Sijie Guo is the tech lead of Twitter’s Messaging group. Sijie is the cocreator of Apache DistributedLog and the PMC chair of Apache BookKeeper.

Photo of Maosong Fu

Maosong Fu


Maosong Fu is the technical lead for ​Heron and ​real-time analytics at Twitter. He ​is the author of ​few publications in distributed area​. Maosong holds a master’s degree from Carnegie Mellon University and a bachelor’s from Huazhong University of Science and Technology.



WeChat QRcode


Stay Connected Image 1
Stay Connected Image 3
Stay Connected Image 2


ORB Data Site