Put open source to work
July 16–17, 2018: Training & Tutorials
July 18–19, 2018: Conference
Portland, OR

Streaming microservices with Akka Streams and Kafka Streams

Boris Lublinsky (Lightbend)
9:00am12:30pm Monday, July 16, 2018
SMACK stack
Location: E145/146
Level: Intermediate
Average rating: **...
(2.23, 13 ratings)

Who is this presentation for?

  • Developers and architects

Prerequisite knowledge

  • A working knowledge of Scala or Java

Materials or downloads needed in advance

  • A laptop with IntelliJ and Scala installed
  • A GitHub account
  • In preparation for tutorial, please download the project from GitHub.

What you'll learn

  • Learn how to use microservices to implement streaming applications based on Akka Streams and Kafka Streams
  • Understand how this approach compares to streaming engines like Spark Streaming, Flink, and Beam

Description

If you’re building streaming data apps, your first inclination might be to reach for Spark Streaming, Flink, Beam, or similar tools, which run as services to which you submit jobs for execution. But sometimes, writing conventional microservices with embedded stream processing is a better fit for your needs.

Kafka Streams and Akka Streams are both libraries that you integrate into your microservices, which means you must manage their lifecycles yourself, but you also get lots of flexibility to do this as you see fit. Kafka Streams is purpose-built for reading data from Kafka topics, processing it, and writing the results to new topics. With powerful stream and table abstractions and an exactly once capability, it supports a variety of common scenarios involving transformation, filtering, and aggregation. Akka Streams, on the other hand, emerged as a dataflow-centric abstraction for the Akka Actors model, designed for general-purpose microservices, especially when per-event low latency is important, such as for complex event processing, where each event requires individual handling. Because of its general-purpose nature, Akka Streams supports a wider class of application problems and third-party integrations, but it’s less focused on Kafka-specific capabilities.

Using an example of model serving, Boris Lublinsky walks you through building streaming apps as microservices using Akka Streams and Kafka Streams. Along the way, Boris discusses the strengths and weaknesses of each tool for particular design needs and contrasts them with Spark Streaming and Flink, so you’ll know when to choose them instead.

Photo of Boris Lublinsky

Boris Lublinsky

Lightbend

Boris Lublinsky is a software architect at Lightbend, where he specializes in big data, stream processing, and services. Boris has over 30 years’ experience in enterprise architecture. Over his career, he has been responsible for setting architectural direction, conducting architecture assessments, and creating and executing architectural roadmaps in fields such as big data (Hadoop-based) solutions, service-oriented architecture (SOA), business process management (BPM), and enterprise application integration (EAI). Boris is the coauthor of Applied SOA: Service-Oriented Architecture and Design Strategies, Professional Hadoop Solutions, and Serving Machine Learning Models. He is also cofounder of and frequent speaker at several Chicago user groups.

Comments on this page are now closed.

Comments

Picture of Boris Lublinsky
Boris Lublinsky | ARCHITECT
07/14/2018 11:49am PDT

In preparation for tutorial, please download the project from GitHub – https://github.com/lightbend/kafka-with-akka-streams-kafka-streams-tutorial