Sep 23–26, 2019

Real-time SQL stream processing at scale with Apache Kafka and KSQL

Viktor Gamov (Confluent)
9:00am12:30pm Tuesday, September 24, 2019
Location: 1E 10

Who is this presentation for?

  • Data engineers, developers, and database administrators

Level

Intermediate

Description

If you’ve ever thought you needed to be a programmer to do stream processing and build stream processing data pipelines, think again. Apache Kafka is a distributed, scalable, and fault-tolerant streaming platform, providing low-latency pub/sub messaging coupled with native storage and stream processing capabilities. Integrating Kafka with a relational database management system (RDBMS), NoSQL, and object stores is simple with Kafka Connect, which is part of Apache Kafka. KSQL is the open source SQL streaming engine for Apache Kafka and makes it possible to build stream processing applications at scale, written using a familiar SQL interface.

Viktor Gamov walks you through the architectural reasoning for Apache Kafka and the benefits of real-time integration. You’ll build a streaming data pipeline using nothing but your bare hands, Kafka Connect, and KSQL.

Prerequisite knowledge

  • A basic understanding of SQL, databases, Linux and Shell, and Docker

Materials or downloads needed in advance

  • A laptop
  • Complete the "setup instructions":https://github.com/confluentinc/quickstart-demos/blob/5.0.0-post/ksql-workshop/pre-requisites.adoc

What you'll learn

  • Discover best practices around building pipelines with Apache Kafka
  • Learn how to use just config and SQL to build complete ETL pipelines
  • Identify patterns for integration with databases and anti-patterns to be aware of
Photo of Viktor Gamov

Viktor Gamov

Confluent

Viktor Gamov is a Developer Advocate at Confluent, the company that makes an event streaming platform based on Apache Kafka. Back in his consultancy days, Viktor developed comprehensive expertise in building enterprise application architectures using open source technologies. He enjoys helping architects and developers to design and develop low latency, scalable and highly available distributed systems. He is a professional conference speaker on distributed systems, streaming data, JVM and DevOps topics, and is regular on events including JavaOne, Devoxx, OSCON, QCon, and others. He co-authored O’Reilly’s «Enterprise Web Development.» He blogs at (http://gamov.io)[gamov.io] and co-hosts «Crazy Russians in Devoops» and «DevRelRad.io» podcasts. Follow Viktor on Twitter @gamussa, where he posts there about gym life, food, open source, and, of course, Kafka and Confluent!

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

strataconf@oreilly.com

For information on exhibiting or sponsoring a conference

Contact list

View a complete list of Strata Data Conference contacts