Sep 23–26, 2019

Managing your Kafka in an explosive growth environment

Alon Gavra (AppsFlyer)
1:15pm1:55pm Thursday, September 26, 2019
Location: 1A 15/16
Average rating: *****
(5.00, 10 ratings)

Who is this presentation for?

  • Architects, DevOps engineers, and application developers

Level

Intermediate

Description

Frequently, Kafka is just a piece of the stack that lives in production that often times no one wants to touch—because it just works. Kafka sits at the core of AppsFlyer’s infrastructure that processes billions of events daily.

Alon Gavra dives into how AppsFlyer built its microservices architecture with Kafka as its core piece to support 70B+ requests daily. With continuous growth, the company needed to “learn on the job” how to improve its Kafka architecture by moving to the producer-owner cluster model, breaking up its massive monolith clusters to smaller, more robust clusters and migrating from an older version of Kafka with real-time production clients and data streams. Alon outlines best practices for leveraging Kafka’s in-memory capabilities and built-in partitioning, as well as some of the tweaks and stabilization mechanisms that enable real-time performance at web scale, alongside processes for continuous upgrades and deployments with end-to-end automation, in an environment of constant traffic growth.

Prerequisite knowledge

  • A basic understanding of Kafka

What you'll learn

  • Learn why Kafka is useful for handling traffic spikes and ensuring no data loss in constant growth
Photo of Alon Gavra

Alon Gavra

AppsFlyer

Alon Gavra is a platform team lead at AppsFlyer. Originally a backend developer, he’s transitioned to lead the real time infrastructure team and took on the role of bringing some of the most heavily used infrastructure in AppsFlyer to the next level. A strong believer in sleep-driven design, Alon’s main focus is stability and resiliency in building massive data ingestion and storage solutions.

    Contact us

    confreg@oreilly.com

    For conference registration information and customer service

    partners@oreilly.com

    For more information on community discounts and trade opportunities with O’Reilly conferences

    strataconf@oreilly.com

    For information on exhibiting or sponsoring a conference

    pr@oreilly.com

    For media/analyst press inquires