Driven by a need for scalability, CoScale recently converted part of its backend to a streaming architecture using Kafka Streams, an open source library for writing streaming applications, which ships with Kafka. Kafka Streams is a relatively new addition to available streaming frameworks (Storm, Samza, Flink, Spark Streaming, etc.), but it has some unique features that make a nice fit for the challenges faced.
Bart De Vylder makes the case for Kafka Streams by comparing the main design decisions to other streaming frameworks. Along the way, he shares his experience converting an existing codebase to Kafka Streams and migrating an existing production environment, highlighting aspects that worked remarkably well as well as the challenges he ran into and the workarounds he used to solve them.
Bart De Vylder is a data scientist at CoScale. Previously, Bart was active in software engineering and architecture, with a focus on distributed systems. His interests lie in machine learning and building reliable, scalable data processing systems. Bart holds a PhD in artificial intelligence from the Free University of Brussels.
©2018, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org