Engineering the Future of Software
Feb 25–26, 2018: Training
Feb 26–28, 2018: Tutorials & Conference
New York, NY

Apache Kafka + Apache Mesos = Highly scalable streaming microservices

Kai Wähner (Confluent)
4:50pm–5:40pm Tuesday, February 27, 2018
Secondary topics:  Framework-focused, Hands-on

Who is this presentation for?

  • Architects, consultants, developers, and project leads

Prerequisite knowledge

  • Familiarity with distributed systems and containers

What you'll learn

  • Explore a highly scalable, mission-critical infrastructure using Apache Kafka and Apache Mesos


Microservices establish many benefits like Agile, flexible development, and deployment of business logic. However, a microservice architecture also creates many new challenges, including increased communication between distributed instances, the need for orchestration, new failover requirements, and resiliency design patterns.

Kai Wähner offers an overview of a highly scalable, mission-critical infrastructure using Apache Kafka and Apache Mesos: Kafka brokers are used as the distributed messaging backbone; Kafka’s Streams API embeds stream processing into any external application without the need for a dedicated streaming cluster; and Mesos is used as a scalable infrastructure to leverage the benefits of a cloud-native platform like service discovery, health checks, and failover management. Kai ends with a live demo that shows how to develop real-time applications for your core business with Kafka messaging brokers and Kafka Streams API and how to deploy, manage, and scale them on a Mesos cluster using different deployment options.

Photo of Kai Wähner

Kai Wähner


Kai Waehner is a technology evangelist at Confluent. Kai’s areas of expertise include big data analytics, machine learning, deep learning, messaging, integration, microservices, the internet of things, stream processing, and the blockchain. He is regular speaker at international conferences such as JavaOne, O’Reilly Software Architecture, and ApacheCon and has written a number of articles for professional journals. Kai also shares his experiences with new technologies on his blog.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)