Sep 23–26, 2019

Kafka/SMM(Streams Messaging Manager) Crash Course

Purnima Reddy Kuchikulla (Cloudera), Dan Chaffelson (Cloudera)
1:30pm5:00pm Tuesday, September 24, 2019
Location: 1E 14
Secondary topics:  Deep dive into specific tools, platforms, or frameworks, Streaming and IoT

Who is this presentation for?

DevOps/developers, platform team and security/governance teams.

Level

Beginner

Description

This session will cover learning about fundamentals of Apache Kafka and the related SMM. Start with understanding the basic concepts/entities of Apache Kafka like Brokers, Topics, Producers and Consumers/Consumer Groups. It will then delve deeper in to advanced topics like idempotent producer, transactional API in Kafka for exactly once processing, authentication, authorization, replication, log compaction, compression, performance, etc. It will later on be followed by a demo of SMM, an open source Hortonworks initiative to help users of Kafka get a better insight in to their Kafka clusters from an operational perspective using an elegant and slick GUI rather than writing complex manual scripts. It will also cover a demo of Alerting/Notification framework that can be used to trigger alerts and notify based on certain conditions one wants to monitor for.

The objective of this session is to learn about Apache Kafka and illustrate how SMM can help to answer questions that arise in production deployments. Example questions are “Do I have any offline topic partitions”, “Which consumer group is falling behind most”, “What producers are generating the most data right now”, “How does data in my application topic look like” and so on. It will also aim to get familiar with SMM GUI exploring different views around different entities like Brokers, Topics, Producers and Consumer Groups so that user can quickly look for valuable information needed to monitor Kafka clusters or their application. It will also aim to learn how to use the Alerting and Notification framework that comes with SMM to automate monitoring of Kafka clusters and the applications built around it.

Prerequisite knowledge

Basic understanding of Distributed Messaging Systems

Materials or downloads needed in advance

laptop with a browser like Chrome

What you'll learn

Attendees will walk away with an understanding of key Kafka concepts. A confidence to better handle Kafka environments and knowledge of how to simplify Kafka operations.
Photo of Purnima Reddy Kuchikulla

Purnima Reddy Kuchikulla

Cloudera

Purnima is a Big Data evangelist with 15 years of experience in the industry. Purnima comes to Cloudera after working with IBM and ADP. She works with customers on their Cloud and Big Data strategies.

Dan Chaffelson

Cloudera

Throughout a decade of virtualisation and launching two startups, Dan has now been nerdy on three continents and in every line of business from UK bulge bracket banking to Australian desert public services.
Joining Hortonworks as a Solutions Engineer in 2016, he swiftly automated a sales manager using Apache Nifi and now drives the international practice for enterprise adoption and automation of the HDF product line, and maintains a public project for Apache NiFi python automation (NiPyAPI) on github.
Dan is based in London with his family and pet Samoyed, he can most recently be found building an open source baby monitor out of Raspberry Pi’s while mining Cryptocurrency in his shed.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

strataconf@oreilly.com

For information on exhibiting or sponsoring a conference

Contact list

View a complete list of Strata Data Conference contacts