Presented By O'Reilly and Cloudera
Make Data Work
Dec 4–5, 2017: Training
Dec 5–7, 2017: Tutorials & Conference
Singapore

In-Person Training
Real-time systems with Spark Streaming and Kafka

Jesse Anderson (Big Data Institute)
Monday, December 4 & Tuesday, December 5, 9:00am - 5:00pm
Big data and the cloud
Location: 335

Participants should plan to attend both days of this 2-day training course. Platinum and Training passes do not include access to tutorials on Tuesday.

To handle real-time big data, you need to solve two difficult problems: how do you ingest that much data, and how will you process that much data? Jesse Anderson explores the latest real-time frameworks (both open source and managed cloud services), discusses the leading cloud providers, and explains how to choose the right one for your company.

What you'll learn, and how you can apply it

  • Learn how to how to ingest data, process it, analyze it, and display it in real time with a dashboard with Apache Kafka and Apache Spark

Prerequisites:

  • A working knowledge of HDFS and Spark (i.e., Spark batch APIs)

Real-time big data frameworks are enabling brand-new use cases, while the cloud is letting us do things cheaper and faster than ever. Together, they’re making it easier to create production real-time systems. But to handle real-time big data, you need to solve two difficult problems: how do you ingest that much data, and how will you process that much data?

Jesse Anderson explores the latest real-time frameworks (both open source and managed cloud services), discusses the leading cloud providers, and explains how to choose the right one for your company. Focusing on Apache Kafka and Apache Spark, Jesse also demonstrates how to ingest data, process it, analyze it, and display it in real time with a dashboard.

For the final exercise, you’ll take data that has been ingested with Kafka and process it with Spark Streaming and visualize it on a web page with D3. This video gives a little more information about the final exercise so you can see the skills you’ll take away from the class.

About your instructor

Photo of Jesse Anderson

Jesse Anderson is a data engineer, creative engineer, and managing director of the Big Data Institute. Jesse trains employees on big data—including cutting-edge technology like Apache Kafka, Apache Hadoop, and Apache Spark. He has taught thousands of students at companies ranging from startups to Fortune 100 companies the skills to become data engineers. He is widely regarded as an expert in the field and recognized for his novel teaching practices. Jesse is published by O’Reilly and Pragmatic Programmers and has been covered in such prestigious media outlets as the Wall Street Journal, CNN, BBC, NPR, Engadget, and Wired. You can learn more about Jesse at Jesse-Anderson.com.

Twitter for jessetanderson

Conference registration

Get the Platinum pass or the Training pass to add this course to your package. .

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Comments

Picture of Jesse Anderson
Jesse Anderson | MANAGING DIRECTOR
10/17/2017 5:29pm SGT

We’ll be focusing on Java as the implementation code. We’ll be both a VM and cloud instances.

This class isn’t an introduction to Big Data or Spark batch. You will need to know those or related technologies. You won’t need any previous Apache Spark or Apache Kafka experience.

Ankit Kansal | SENIOR DATA ENGINEER
10/13/2017 8:14pm SGT

We want to know, what technology stack (language scala, python or java ?) and environment (local instance / Databricks e.t.c ?) will you be using in the training program for spark streaming program. And how much level of expertise will be required in each of them as well ?