Build Systems that Drive Business
June 11–12, 2018: Training
June 12–14, 2018: Tutorials & Conference
San Jose, CA

Distributed systems for stream processing: Apache Kafka and Spark Streaming

Lena Hall (Microsoft)
1:15pm–1:55pm Wednesday, June 13, 2018
Distributed Data
 Location: 230 B Level: Intermediate
Secondary topics: Systems Architecture & Infrastructure
Average rating: ****.
(4.12, 8 ratings)

Prerequisite knowledge

  • A basic understanding of data processing

What you'll learn

  • Learn how to use distributed systems like Apache Kafka and Spark Streaming

Description

Everything is a data source. Today’s online activities, financial operations, IoT devices, and sensors generate data at an ever-increasing rate, so your architecture for ingesting these incoming influxes of data needs to be flexible, scalable, fast, and resilient.

Alena Hall walks you through using distributed systems like Apache Kafka and Spark Streaming. You’ll learn how to set up your infrastructure and build a distributed streaming architecture on Azure using open source frameworks like Apache Kafka and Spark Streaming and then use these distributed systems to process data coming from multiple sources in real time, perform machine learning tasks, and learn how to be effective interactively experimenting with streams using code.

Photo of Lena Hall

Lena Hall

Microsoft

Lena Hall is a senior software engineer and developer advocate at Microsoft working on Azure, where she focuses on large-scale distributed systems and modern architectures. Lena has more than 10 years of experience in software engineering with a focus on distributed cloud programming, real-time system design, highly scalable and performant systems, big data analysis, data science, functional programming, and machine learning. Previously, she was a senior software engineer at Microsoft Research. She’s an elected member of the F# Software Foundation’s board of trustees, co-organizes a conference called ML4ALL, and is often an invited member of program committees for conferences like Kafka Summit, Lambda World, and others. Lena holds a master’s degree in computer science.

Comments on this page are now closed.

Comments

Picture of Lena Hall
Lena Hall | SENIOR SOFTWARE ENGINEER
06/18/2018 5:38am PDT

Thanks Eric. Yes, I’ll be writing up a detailed article based on the talk with all the information in it. I’ll post an update when it’s published!

Picture of Eric Bach
Eric Bach | SOLUTION ENGINEER
06/18/2018 5:32am PDT

I found you session very valuable. When do you expect the content to be available for download?