Presented By O'Reilly and Cloudera
Make Data Work
September 26–27, 2016: Training
September 27–29, 2016: Tutorials & Conference
New York, NY

Implementing streaming architecture with Apache Flink: Present and future

Kostas Tzoumas (data Artisans)
2:05pm–2:45pm Thursday, 09/29/2016
IoT & real-time
Location: 3D 08 Level: Intermediate
Tags: real-time
Average rating: ****.
(4.00, 2 ratings)

Prerequisite knowledge

  • A general understanding of the concept of data streaming as well as familiarity with Flink, Kafka, or related technologies
  • What you'll learn

  • Learn the basics of Apache Flink from a user perspective
  • Explore the latest developments and the directions of the Flink project
  • Description

    Apache Flink has seen incredible growth during the last year, both in development and usage, driven, to a large extent, by the fundamental shift in the enterprise from batch to stream processing. A streaming-first architecture enables continuous processing on data that is continuously produced (as it is in most interesting datasets), enabling real-time decisions but also a simplified architecture that can subsume batch processing. Kostas Tzoumas dives into the benefits of using Flink as the central piece of such architecture. In addition, Kostas covers the latest developments in the project and the future roadmap, such as the ability to query the state in the stream processor, new libraries (e.g., SQL and CEP), dynamic scaling, seamless application and Flink updates, and integration between batch and streaming, which leads to radically simplified architecture and deployment. Kostas concludes with a sample of what production users of Flink are currently achieving with the system.

    Topics include:

    • Simplifying the data processing stack with a microservices-based architecture while at the same time enabling real-time decisions
    • Processing very-high-volume streams (millions of events per second) with very low latency efficiently (in small clusters)
    • How Flink is unique in its ability to enable accurate, fault-tolerant, repeatable, and practical production deployment of stateful streaming applications with its support for exactly once processing, event time, and savepoints
    Photo of Kostas Tzoumas

    Kostas Tzoumas

    data Artisans

    Kostas Tzoumas is a PMC member of the Apache Flink project and cofounder of data Artisans, the company founded by the original development team that created Flink. Kostas has spoken extensively about Flink, including at Hadoop Summit San Jose 2015.