Presented By O'Reilly and Cloudera
Make Data Work
Dec 4–5, 2017: Training
Dec 5–7, 2017: Tutorials & Conference
Singapore

Top five mistakes when writing streaming applications

Ted Malaska (Blizzard Entertainment)
11:15am11:55am Wednesday, December 6, 2017
Data engineering and architecture
Location: 308/309 Level: Intermediate

Who is this presentation for?

  • Software engineers, tech leads, and architects

Prerequisite knowledge

  • A basic understanding of the streaming ecosystem

What you'll learn

  • Learn best practices for writing streaming applications

Description

Ted Malaska shares the top five mistakes that no one talks about when you start writing your streaming app along with the practices you’ll inevitably need to learn along the way.

Topics include:

  • How do I manage offsets?
  • How do I manage state?
  • How do I make my Spark streaming job resilient to failures? Can I avoid some failures?
  • How do I gracefully shut down my streaming job?
  • How do I monitor and manage (e.g., retry logic) streaming job?
  • How can I better manage the DAG in my streaming job?
  • When do I use checkpointing and for what? When shouldn’t I use checkpointing?
  • Do I need a WAL when using a streaming data source? Why? When don’t I need one?
Photo of Ted Malaska

Ted Malaska

Blizzard Entertainment

Ted Malaska is a group technical architect on the Battle.net team at Blizzard, helping support great titles like World of Warcraft, Overwatch, and HearthStone. Previously, Ted was a principal solutions architect at Cloudera, helping clients find success with the Hadoop ecosystem, and a lead architect at the Financial Industry Regulatory Authority (FINRA). He has also contributed code to Apache Flume, Apache Avro, Apache Yarn, Apache HDFS, Apache Spark, Apache Sqoop, and many more. Ted is a coauthor of Hadoop Application Architectures, a frequent speaker at many conferences, and a frequent blogger on data architectures.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)