Apache Flink is evolving from a framework for streaming data analytics to a platform that offers a foundation for event-driven applications. Over the past year, an increasing number of users have put Flink into the center of their business logic and entrusted it with their most valuable assets: their application data. For example, entire social networks have been built on top of Flink, using Flink to replace data management aspects that are typically handled by a database in more conventional architectures.
Powering this evolution is Flink’s sophisticated state management features and its streams-and-snapshots approach to stateful stream processing. Tzu-Li (Gordon) Tai explores Flink’s stateful stream processing runtime, where it fits in when building event-driven applications, and how it can even replace databases, highlighting key features like consistent point-in-time savepoints, event-time awareness, flexible rescaling, extremely large state handling, state (schema) evolution, and queryable state. Along the way, Gordon also demonstrates these features in action.
Tzu-Li (Gordon) Tai is a software engineer at data Artisans and an Apache Flink committer and PMC member. His main contributions on Flink include work on Flink’s streaming connectors (Kafka, AWS Kinesis, Elasticsearch) and its type serialization stack and state management capabilities. Gordon is a frequent speaker at conferences such as Flink Forward, Flink meetups in Berlin and Taiwan, and several Taiwan-based conferences on the Hadoop ecosystem and data engineering in general.
©2017, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org