Presented By
O’Reilly + Cloudera
Make Data Work
March 25-28, 2019
San Francisco, CA

Put Kafka in jail with Strimzi

Sean Glover (Lightbend)
4:20pm5:00pm Wednesday, March 27, 2019
Secondary topics:  Streaming, realtime analytics, and IoT
Average rating: ****.
(4.00, 1 rating)

Who is this presentation for?

  • DevOps engineers, software engineers, and operations engineers

Level

Beginner

Prerequisite knowledge

  • Familiarity with Kubernetes and Kafka

What you'll learn

  • Learn about Strimzi and the Kubernetes Operator pattern for creating smart infrastructure
  • Discover how to easily run Kafka and ZooKeeper clusters for Kubernetes

Description

Kafka is best suited to run close to the metal on dedicated machines in statically defined clusters, but static clusters are becoming less common. Companies want to create mixed-use clusters that take advantage of every resource available. Stateless, transient services fit well into this model, but complex stateful services each have their own particular needs.

Strimzi is a leading open source, multi-Operator-based project to run Apache Kafka and ZooKeeper on Kubernetes. Sean Glover discusses the pros and cons of running a Strimzi-operated Kafka cluster relative to installing it directly on the host platform. Static clusters require greater operational knowhow to do common tasks with Kafka, such as applying broker configuration updates, upgrading to a new version, and adding or decommissioning brokers. Strimzi makes many Kafka operational tasks easy, such as initial deployment of a Kafka and ZooKeeper cluster, config-based topic management, persistent volume management, zero-downtime cluster updates and upgrades, scaling brokers, moving brokers, running multiple Kafka clusters on the same cluster, and multi-data center support in active-active and active-passive configurations.

Kafka is an integral part of Lightbend’s next-generation stream processing system the Fast Data Platform, and Strimzi is an important component that facilitates running the Fast Data Platform on Kubernetes.

Photo of Sean Glover

Sean Glover

Lightbend

Sean Glover is a senior software engineer on the Fast Data Platform team at Lightbend, where he specializes in Kubernetes, Apache Kafka, and its ecosystem. Sean enjoys building fast data platforms and reactive distributed systems and contributing to open source projects.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)