Build & maintain complex distributed systems
October 1–2, 2017: Training
October 2–4, 2017: Tutorials & Conference
New York, NY

Running a massively parallel stream processing system at Netflix

Zhenzhong Xu (Netflix)
1:30pm2:10pm Wednesday, October 4, 2017
Distributed Data & Databases, Real time, events, streams & scale
Location: Grand Ballroom West Level: Advanced
Average rating: *****
(5.00, 2 ratings)

Who is this presentation for?

  • Software engineers, architects, data engineers, and those working in data infrastructure

Prerequisite knowledge

  • A basic understanding of distributed systems, stream processing, and cloud-native microservices architectures
  • A working knowledge of Apache Kafka (or another similar replayable streaming source)

What you'll learn

  • Learn how Netflix approaches big data streaming infrastructure, from high-level architecture to operations and tools in the ecosystem

Description

Over 200 million devices worldwide are capable of streaming Netflix content. Sitting on top of a microservice architecture, the entire ecosystem generates more than a trillion events each day to feed critical Netflix systems to monitor service health, detect fraudulent behaviors, improve customer experience, etc.

Keystone, a critical piece of Netflix’s backend data infrastructure, ensures a massive amount of events are delivered in near real time reliably, at scale, and in the face of failures. Zhenzhong Xu leads a deep dive into Keystone’s architecture and underlying stream processing engines, sharing insights and proven paths on how the company achieves multitenancy, scalability, and resilience in a complex cloud-native distributed system environment.

Photo of Zhenzhong Xu

Zhenzhong Xu

Netflix

Zhenzhong Xu is a software engineer working on highly scalable and resilient streaming data infrastructure at Netflix. Previously, he was a core contributor to Microsoft Azure data center operating system reconciliation management and resiliency functionalities. He is passionate about anything related to real-time data systems and large-scale distributed systems.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)