Mar 15–18, 2020

Professional Kafka development (Day 2)

Jesse Anderson (Big Data Institute)
Location: 232

Who is this presentation for?

Data engineers, data architects, developers




Day 1

  • Data at scale
  • Data movement concepts
  • Moving data at scale
  • Kafka concepts
  • Kafka system
  • Basic concepts
  • Advanced concepts
  • Developing with Kafka
  • Using Apache Maven
  • Kafka APIs
  • Kafka API caveats
  • Advanced Kafka development
  • Advanced consumers and producers
  • Advanced offset handling
  • Transactions
  • Multithreading consumers

Day 2

  • Kafka and Avro
  • Why serialize
  • Avro and serialization formats
  • Kafka Connect
  • Using Kafka Connect
  • Importing from JDBC
  • Exporting to HDFS
  • Kafka Streams
  • Kafka Streams
  • The Kafka Streams API
  • KSQL
  • Using KSQL
  • Wrap-up and Q&A

Prerequisite knowledge

  • A working knowledge of programming in Java
  • A basic understanding of big data technologies (useful but not required)

What you'll learn

  • Learn how to create large-scale real-time systems using Apache Kafka, how to create Kafka producers and consumers, and how to use Kafka Streams to create data pipelines
  • Understand how real-time distributed systems are different from batch systems and how to configure Kafka Connect to put data into and take data out of Kafka
  • Discover how to use Apache Avro with Kafka for truly enterprise-grade solutions and how to use your existing SQL skills with KSQL
  • Explore best practices and common architectural patterns when creating solutions with Kafka
Photo of Jesse Anderson

Jesse Anderson

Big Data Institute

Jesse Anderson is a big data engineering expert and trainer.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

For conference registration information and customer service

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

For media/analyst press inquires