Kafka and Streams Messaging Manager (SMM) crash course
Who is this presentation for?
- DevOps practitioners, developers, platform team, and security and governance teams
Purnima Reddy Kuchikulla and Dan Chaffelson cover the fundamentals of Apache Kafka and the related Streams Messaging Manager (SMM). They explore the basic concepts and entities of Apache Kafka, like brokers, topics, producers, and consumers and consumer groups. And they delve deeper into advanced topics like idempotent producer, transactional API in Kafka for exactly once processing, authentication, authorization, replication, log compaction, compression, performance, etc. You’ll get a demo of SMM, an open source Hortonworks initiative to help users of Kafka get a better insight into their Kafka clusters from an operational perspective using an elegant and slick GUI rather than writing complex manual scripts. You’ll also see a demo of an alerting and notification framework that can be used to trigger alerts and notify you based on certain conditions you want to monitor for.
You’ll learn about Apache Kafka and see how SMM can help answer questions that arise in production deployments, such as identifying offline topic partitions, questions about your consumer groups, data production, what certain data looks like, and so on. You’ll also get familiar with SMM GUI by exploring different views around different entities like brokers, topics, producers, and consumer groups so that you can quickly look for valuable information needed to monitor Kafka clusters or your application. And Purnima and Dan detail how to use the alerting and notification framework that comes with SMM to automate monitoring of Kafka clusters and the applications built around it.
- A basic understanding of distributed messaging systems
Materials or downloads needed in advance
- A laptop with a browser like Chrome installed
What you'll learn
- Gain an understanding of key Kafka concepts
- Develop the confidence to better handle Kafka environments
- Learn how to simplify Kafka operations
Purnima Reddy Kuchikulla
Purnima Kuchikulla is a solution engineer at Cloudera, where she works with customers on their cloud and big data strategies, and a big data evangelist with 15 years of experience in the industry. Previously, she was at IBM and ADP.
Dan Chaffelson is a director of DataFlow field engineering at Cloudera. Previously, Dan was a solutions engineer at Hortonworks. He drives the international practice for enterprise adoption of the HDF product line and maintains a public project for Apache NiFi Python automation (NiPyAPI) on GitHub. Throughout a decade of virtualization and launching two startups, Dan has been nerdy on three continents and in every line of business from UK bulge bracket banking to Australian desert public services. Dan is based in London with his family and pet samoyed; he can be found building an open source baby monitor out of Raspberry Pi’s while mining cryptocurrency in his shed.
Tony Wu is an engineering manager at Cloudera, where he manages the Altus core engineering team. Previously, Tony was a team lead for the partner engineering team at Cloudera. He’s responsible for Microsoft Azure integration for Cloudera Director.
Leave a Comment or Question
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
For conference registration information and customer service
For more information on community discounts and trade opportunities with O’Reilly conferences
For information on exhibiting or sponsoring a conference
For media/analyst press inquires