Presented By O'Reilly and Cloudera
Make Data Work
31 May–1 June 2016: Training
1 June–3 June 2016: Conference
London, UK

When it absolutely, positively has to be there: Reliability guarantees in Kafka

Gwen Shapira (Confluent), Jeff Holoman (Cloudera)
12:05–12:45 Thursday, 2/06/2016
IoT & real-time
Location: Capital Suite 12 Level: Intermediate
Average rating: ****.
(4.33, 3 ratings)

Prerequisite knowledge

Attendees should be familiar with Apache Kafka.


In the financial industry, losing data is unacceptable. Financial firms are adopting Kafka for their critical applications because Kafka provides the low latency, high throughput, high availability, and scale that these applications require. But can it also provide complete reliability? Gwen Shapira and Jeff Holoman go over everything that happens to a message—from producer to consumer—and pinpoint all the places where data can be lost if you’re not careful. Gwen and Jeff explain how developers and operation teams can work together to build a bulletproof data pipeline with Kafka and demonstrate how to build the system to prove the reliability of your pipeline.

Photo of Gwen Shapira

Gwen Shapira


Gwen Shapira is a system architect at Confluent, where she helps customers achieve success with their Apache Kafka implementations. She has 15 years of experience working with code and customers to build scalable data architectures, integrating relational and big data technologies. Gwen currently specializes in building real-time reliable data processing pipelines using Apache Kafka. Gwen is an Oracle Ace Director, the coauthor of Hadoop Application Architectures, and a frequent presenter at industry conferences. She is also a committer on Apache Kafka and Apache Sqoop. When Gwen isn’t coding or building data pipelines, you can find her pedaling her bike, exploring the roads and trails of California and beyond.

Photo of Jeff Holoman

Jeff Holoman


Jeff Holoman is a systems engineer at Cloudera. Jeff is a Kafka contributor and has focused on helping customers with large-scale Hadoop deployments, primarily in financial services. Prior to his time at Cloudera, Jeff worked as an application developer, system administrator, and Oracle technology specialist.