Sep 23–26, 2019

Kafka and Streams Messaging Manager (SMM) crash course

Purnima Reddy Kuchikulla (Cloudera), Dan Chaffelson (Cloudera)
1:30pm5:00pm Tuesday, September 24, 2019
Location: 1E 14

Who is this presentation for?

  • DevOps practitioners, developers, platform team, and security and governance teams

Level

Beginner

Description

Purnima Reddy Kuchikulla and Dan Chaffelson cover the fundamentals of Apache Kafka and the related Streams Messaging Manager (SMM). They explore the basic concepts and entities of Apache Kafka, like brokers, topics, producers, and consumers and consumer groups. And they delve deeper into advanced topics like idempotent producer, transactional API in Kafka for exactly once processing, authentication, authorization, replication, log compaction, compression, performance, etc. You’ll get a demo of SMM, an open source Hortonworks initiative to help users of Kafka get a better insight into their Kafka clusters from an operational perspective using an elegant and slick GUI rather than writing complex manual scripts. You’ll also see a demo of an alerting and notification framework that can be used to trigger alerts and notify you based on certain conditions you want to monitor for.

You’ll learn about Apache Kafka and see how SMM can help answer questions that arise in production deployments, such as identifying offline topic partitions, questions about your consumer groups, data production, what certain data looks like, and so on. You’ll also get familiar with SMM GUI by exploring different views around different entities like brokers, topics, producers, and consumer groups so that you can quickly look for valuable information needed to monitor Kafka clusters or your application. And Purnima and Dan detail how to use the alerting and notification framework that comes with SMM to automate monitoring of Kafka clusters and the applications built around it.

Prerequisite knowledge

  • A basic understanding of distributed messaging systems

Materials or downloads needed in advance

  • A laptop with a browser like Chrome installed

What you'll learn

  • Gain an understanding of key Kafka concepts
  • Develop the confidence to better handle Kafka environments
  • Learn how to simplify Kafka operations
Photo of Purnima Reddy Kuchikulla

Purnima Reddy Kuchikulla

Cloudera

Purnima Kuchikulla is a solution engineer at Cloudera, where she works with customers on their cloud and big data strategies, and a big data evangelist with 15 years of experience in the industry. Previously, she was at IBM and ADP.

Dan Chaffelson

Cloudera

Dan Chaffelson is a director of DataFlow field engineering at Cloudera. Previously, Dan was a solutions engineer at Hortonworks. He drives the international practice for enterprise adoption of the HDF product line and maintains a public project for Apache NiFi Python automation (NiPyAPI) on GitHub. Throughout a decade of virtualization and launching two startups, Dan has been nerdy on three continents and in every line of business from UK bulge bracket banking to Australian desert public services. Dan is based in London with his family and pet samoyed; he can be found building an open source baby monitor out of Raspberry Pi’s while mining cryptocurrency in his shed.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

strataconf@oreilly.com

For information on exhibiting or sponsoring a conference

Contact list

View a complete list of Strata Data Conference contacts