Build & maintain complex distributed systems
October 1–2, 2017: Training
October 2–4, 2017: Tutorials & Conference
New York, NY

Running a massively parallel stream processing system at Netflix

Zhenzhong Xu (Netflix)
1:30pm2:10pm Wednesday, October 4, 2017
Distributed Data & Databases, Real time, events, streams & scale
Location: Grand Ballroom West Level: Advanced
Average rating: *****
(5.00, 2 ratings)

Who is this presentation for?

  • Software engineers, architects, data engineers, and those working in data infrastructure

Prerequisite knowledge

  • A basic understanding of distributed systems, stream processing, and cloud-native microservices architectures
  • A working knowledge of Apache Kafka (or another similar replayable streaming source)

What you'll learn

  • Learn how Netflix approaches big data streaming infrastructure, from high-level architecture to operations and tools in the ecosystem

Description

Over 200 million devices worldwide are capable of streaming Netflix content. Sitting on top of a microservice architecture, the entire ecosystem generates more than a trillion events each day to feed critical Netflix systems to monitor service health, detect fraudulent behaviors, improve customer experience, etc.

Keystone, a critical piece of Netflix’s backend data infrastructure, ensures a massive amount of events are delivered in near real time reliably, at scale, and in the face of failures. Zhenzhong Xu leads a deep dive into Keystone’s architecture and underlying stream processing engines, sharing insights and proven paths on how the company achieves multitenancy, scalability, and resilience in a complex cloud-native distributed system environment.

Photo of Zhenzhong Xu

Zhenzhong Xu

Netflix

Zhenzhong Xu is a software engineer working on highly scalable and resilient streaming data infrastructure at Netflix. Previously, he was a core contributor to Microsoft Azure data center operating system reconciliation management and resiliency functionalities. He is passionate about anything related to real-time data systems and large-scale distributed systems.