Kafka is best suited to run close to the metal on dedicated machines in statically defined clusters, but static clusters are becoming less common. Companies want to create mixed-use clusters that take advantage of every resource available. Stateless, transient services fit well into this model, but complex stateful services each have their own particular needs.
Strimzi is a leading open source, multi-Operator-based project to run Apache Kafka and ZooKeeper on Kubernetes. Sean Glover discusses the pros and cons of running a Strimzi-operated Kafka cluster relative to installing it directly on the host platform. Static clusters require greater operational knowhow to do common tasks with Kafka, such as applying broker configuration updates, upgrading to a new version, and adding or decommissioning brokers. Strimzi makes many Kafka operational tasks easy, such as initial deployment of a Kafka and ZooKeeper cluster, config-based topic management, persistent volume management, zero-downtime cluster updates and upgrades, scaling brokers, moving brokers, running multiple Kafka clusters on the same cluster, and multi-data center support in active-active and active-passive configurations.
Kafka is an integral part of Lightbend’s next-generation stream processing system the Fast Data Platform, and Strimzi is an important component that facilitates running the Fast Data Platform on Kubernetes.
Sean Glover is a senior software engineer on the Fast Data Platform team at Lightbend, where he specializes in Kubernetes, Apache Kafka, and its ecosystem. Sean enjoys building fast data platforms and reactive distributed systems and contributing to open source projects.
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
©2019, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com