Build Systems that Drive Business
June 11–12, 2018: Training
June 12–14, 2018: Tutorials & Conference
San Jose, CA

Distributed systems for stream processing: Apache Kafka and Spark Streaming

Alena Hall (Microsoft)
1:15pm–1:55pm Wednesday, June 13, 2018
Distributed Data
Location: 230 B Level: Intermediate
Secondary topics: Systems Architecture & Infrastructure
Average rating: ****.
(4.12, 8 ratings)

Prerequisite knowledge

  • A basic understanding of data processing

What you'll learn

  • Learn how to use distributed systems like Apache Kafka and Spark Streaming

Description

Everything is a data source. Today’s online activities, financial operations, IoT devices, and sensors generate data at an ever-increasing rate, so your architecture for ingesting these incoming influxes of data needs to be flexible, scalable, fast, and resilient.

Alena Hall walks you through using distributed systems like Apache Kafka and Spark Streaming. You’ll learn how to set up your infrastructure and build a distributed streaming architecture on Azure using open source frameworks like Apache Kafka and Spark Streaming and then use these distributed systems to process data coming from multiple sources in real time, perform machine learning tasks, and learn how to be effective interactively experimenting with streams using code.

Photo of Alena Hall

Alena Hall

Microsoft

Alena Hall is a senior software engineer at Microsoft working on Azure, where she focuses on big data and large-scale distributed systems. Previously, she was a senior software engineer at Microsoft Research. Alena has more than 10 years of experience in the software engineering industry with a focus on distributed cloud programming, real-time system modeling, high load and performance, big data analysis, data science, functional programming, and machine learning. She is an elected member of the F# Software Foundation’s board of trustees. Alena holds a master’s degree in computer science and information technology.

Comments on this page are now closed.

Comments

Picture of Alena Hall
Alena Hall | SENIOR SOFTWARE ENGINEER
06/18/2018 5:38am PDT

Thanks Eric. Yes, I’ll be writing up a detailed article based on the talk with all the information in it. I’ll post an update when it’s published!

Picture of Eric Bach
Eric Bach | SOLUTION ENGINEER
06/18/2018 5:32am PDT

I found you session very valuable. When do you expect the content to be available for download?