Schedule: Scale sessions

Steven Wu explains how Netflix’s SPaaS platform empowers users to focus on extracting insights from data streams and build stream processing applications and shares lessons learned building and operating the largest SPaaS use case: Netflix’s Keystone data pipeline, a self-serve platform for creating near-real-time event pipelines that processes three trillion events and 12 PB of data every day. Read more.

Kai Wähner shares a highly scalable, mission-critical infrastructure using Apache Kafka and Apache Mesos: Kafka brokers are used as the distributed messaging backbone; Kafka’s Streams API embeds stream processing into any external application without the need for a dedicated streaming cluster; and Mesos is used as a scalable infrastructure to leverage the benefits of a cloud-native platform. Read more.

Duncan DeVore discusses the ins and outs of dealing with modular JVM-based application consistency, distributed state, and identity coherence with techniques such as idempotency, eventual and casual consistency, the CAP theorem, single source of truth, and distributed domain design. Read more.

Robert Lefkowitz offers a overview of technical debt, explaining how to prevent or reduce it, when to increase it, and how to use refactoring to refinance it. Read more.

Viktor Klang offers a new take on enterprise integration patterns that builds on top of the Reactive Streams standard, an orchestration layer where transformations are standalone, composable, and reusable and—most importantly—use asynchronous flow-control (backpressure) to maintain predictable, stable behavior over time. Read more.

As Netflix continues its journey beyond 100M members, the company is rearchitecting its critical Playback API service to better serve its business needs for the next three to five years. Suudhan Rangarajan discusses why and how Netflix rebuilt the Playback API service and outlines a rigorous framework that you can use to reason about your microservice architecture. Read more.

In an effort to consolidate and modernize the company’s technology stack, Shutterstock recently embarked on a technology overhaul, which also led to organizational and cultural change. Marty Brodbeck shares some of the decisions Shutterstock made and the challenges it faced during this huge transformation, along with key principles that drove and guided the shift. Read more.