Presented By O’Reilly and Cloudera
Make Data Work
21–22 May 2018: Training
22–24 May 2018: Tutorials & Conference
London, UK

Executive Briefing: What you need to know about fast data

Dean Wampler (Lightbend)
14:5515:35 Wednesday, 23 May 2018

Who is this presentation for?

  • Business executives

What you'll learn

  • Learn the business motivations for fast data applications, the organizational challenges required when moving to streaming architectures, and how what your organization already knows about microservices can meet those challenges

Description

Streaming data systems, so called fast data, promise accelerated access to information, leading to new innovations and competitive advantages. But they aren’t just faster versions of big data. They force architecture changes to meet new demands for reliability and dynamic scalability, more like microservices. Dean Wampler outlines what you need to know to exploit fast data successfully.

Big data started with an emphasis on batch-oriented architectures, where data is captured in large, scalable stores then processed using batch jobs. To reduce the gap between data arrival and information extraction, these architectures are now evolving to be stream oriented, where data is processed as it arrives. While a new buzzword, fast data is also a new opportunity for innovation in how your organization leverages data.

However, fast data architectures introduce new challenges for your organization. Whereas a batch job might run for hours, a stream processing application might run for weeks or months. This raises the bar for making these systems resilient against traffic spikes, hardware and network failures, and so forth. The microservice world has faced these challenge for a while. Your data teams will likely need to evolve to resemble the teams you already have for your microservices-based systems. In fact, you’ll probably merge these teams over time, as your microservices do more data processing and your data systems leverage your microservices.

Topics include:

  • The business justification for transitioning from batch-oriented big data to stream-oriented fast data.
  • The organizational changes that streaming architectures require to meet their higher demands for reliability, resiliency, dynamic scalability, etc.
  • How some of these requirements can be met by leveraging what your organization already knows about microservice architectures.
Photo of Dean Wampler

Dean Wampler

Lightbend

Dean Wampler is the vice president of fast data engineering at Lightbend, where he leads the creation of the Lightbend Fast Data Platform, a distribution of scalable, distributed stream processing tools including Spark, Flink, Kafka, and Akka, with machine learning and management tools. Dean is the author of Programming Scala and Functional Programming for Java Developers and the coauthor of Programming Hive, all from O’Reilly. He is a contributor to several open source projects. A frequent Strata speaker, he’s also the co-organizer of several conferences around the world and several user groups in Chicago.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)