Put Kafka in jail with Strimzi

Description

Kafka is best suited to run close to the metal on dedicated machines in statically defined clusters, but static clusters are becoming less common. Companies want to create mixed-use clusters that take advantage of every resource available. Stateless, transient services fit well into this model, but complex stateful services each have their own particular needs.

Strimzi is a leading open source, multi-Operator-based project to run Apache Kafka and ZooKeeper on Kubernetes. Sean Glover discusses the pros and cons of running a Strimzi-operated Kafka cluster relative to installing it directly on the host platform. Static clusters require greater operational knowhow to do common tasks with Kafka, such as applying broker configuration updates, upgrading to a new version, and adding or decommissioning brokers. Strimzi makes many Kafka operational tasks easy, such as initial deployment of a Kafka and ZooKeeper cluster, config-based topic management, persistent volume management, zero-downtime cluster updates and upgrades, scaling brokers, moving brokers, running multiple Kafka clusters on the same cluster, and multi-data center support in active-active and active-passive configurations.

Kafka is an integral part of Lightbend’s next-generation stream processing system the Fast Data Platform, and Strimzi is an important component that facilitates running the Fast Data Platform on Kubernetes.