Kafka and Streams Messaging Manager (SMM) crash course

Description

Purnima Reddy Kuchikulla and Dan Chaffelson cover the fundamentals of Apache Kafka and the related Streams Messaging Manager (SMM). They explore the basic concepts and entities of Apache Kafka, like brokers, topics, producers, and consumers and consumer groups. And they delve deeper into advanced topics like idempotent producer, transactional API in Kafka for exactly once processing, authentication, authorization, replication, log compaction, compression, performance, etc. You’ll get a demo of SMM, an open source Hortonworks initiative to help users of Kafka get a better insight into their Kafka clusters from an operational perspective using an elegant and slick GUI rather than writing complex manual scripts. You’ll also see a demo of an alerting and notification framework that can be used to trigger alerts and notify you based on certain conditions you want to monitor for.

You’ll learn about Apache Kafka and see how SMM can help answer questions that arise in production deployments, such as identifying offline topic partitions, questions about your consumer groups, data production, what certain data looks like, and so on. You’ll also get familiar with SMM GUI by exploring different views around different entities like brokers, topics, producers, and consumer groups so that you can quickly look for valuable information needed to monitor Kafka clusters or your application. And Purnima and Dan detail how to use the alerting and notification framework that comes with SMM to automate monitoring of Kafka clusters and the applications built around it.