Presented By
O’Reilly + Cloudera
Make Data Work
March 25-28, 2019
San Francisco, CA
N Lu

N Lu
Software Engineer, Twitter

Neng Lu is a software engineer at Twitter, where he is the core member of Twitter’s real-time compute team and the core committer to the Apache Heron project (incubating). He has a broad interest in distributed systems and real-time analytics and has worked on Twitter’s key-value storage system, Manhattan; its monitoring system, Cuckoo; and its real-time processing system, Heron. He holds an MS in CS from UCLA and a bachelor’s degree in CS from Zhejiang University.

Sessions

3:50pm–4:30pm Thursday, March 28, 2019
Real-time monitoring of Twitter's network infrastructure with Heron
Data Engineering & Architecture
 Location: 2024
Secondary topics:  Data Integration and Data Pipelines, Security and Privacy, Streaming, realtime analytics, and IoT
J Delange (Twitter), N Lu (Twitter)
Average rating: **...
(2.67, 3 ratings)
Julien Delange and Neng Lu explain how Twitter uses the Heron stream processing engine to monitor and analyze its network infrastructure—implementing a new data pipeline that ingests multiple sources and processes about 1 billion tuples to detect network issues and generate usage statistics. Join in to learn the key technologies used, the architecture, and the challenges Twitter faced. Read more.