Kafka/SMM(Streams Messaging Manager) Crash Course
Who is this presentation for?DevOps/developers, platform team and security/governance teams.
This session will cover learning about fundamentals of Apache Kafka and the related SMM. Start with understanding the basic concepts/entities of Apache Kafka like Brokers, Topics, Producers and Consumers/Consumer Groups. It will then delve deeper in to advanced topics like idempotent producer, transactional API in Kafka for exactly once processing, authentication, authorization, replication, log compaction, compression, performance, etc. It will later on be followed by a demo of SMM, an open source Hortonworks initiative to help users of Kafka get a better insight in to their Kafka clusters from an operational perspective using an elegant and slick GUI rather than writing complex manual scripts. It will also cover a demo of Alerting/Notification framework that can be used to trigger alerts and notify based on certain conditions one wants to monitor for.
The objective of this session is to learn about Apache Kafka and illustrate how SMM can help to answer questions that arise in production deployments. Example questions are “Do I have any offline topic partitions”, “Which consumer group is falling behind most”, “What producers are generating the most data right now”, “How does data in my application topic look like” and so on. It will also aim to get familiar with SMM GUI exploring different views around different entities like Brokers, Topics, Producers and Consumer Groups so that user can quickly look for valuable information needed to monitor Kafka clusters or their application. It will also aim to learn how to use the Alerting and Notification framework that comes with SMM to automate monitoring of Kafka clusters and the applications built around it.
Prerequisite knowledgeBasic understanding of Distributed Messaging Systems
Materials or downloads needed in advance
What you'll learn
Purnima Reddy Kuchikulla
Purnima is a Big Data evangelist with 15 years of experience in the industry. Purnima comes to Cloudera after working with IBM and ADP. She works with customers on their Cloud and Big Data strategies.
Throughout a decade of virtualisation and launching two startups, Dan has now been nerdy on three continents and in every line of business from UK bulge bracket banking to Australian desert public services.
Joining Hortonworks as a Solutions Engineer in 2016, he swiftly automated a sales manager using Apache Nifi and now drives the international practice for enterprise adoption and automation of the HDF product line, and maintains a public project for Apache NiFi python automation (NiPyAPI) on github.
Dan is based in London with his family and pet Samoyed, he can most recently be found building an open source baby monitor out of Raspberry Pi’s while mining Cryptocurrency in his shed.
Leave a Comment or Question
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
For conference registration information and customer service
For more information on community discounts and trade opportunities with O’Reilly conferences
For information on exhibiting or sponsoring a conference
View a complete list of Strata Data Conference contacts