Presented By O'Reilly and Cloudera
December 5-6, 2016: Training
December 6–8, 2016: Tutorials & Conference
Singapore

Top five mistakes when writing Spark applications

Ted Malaska (Blizzard Entertainment), Mark Grover (Cloudera)
11:15am–11:55am Wednesday, December 7, 2016
Spark & beyond
Location: Summit 2 Level: Intermediate
Average rating: ****.
(4.12, 8 ratings)

Prerequisite Knowledge

  • A working knowledge of Spark

What you'll learn

  • Understand common mistakes and how to fix them, in order to write more capable, powerful Spark applications

Description

In the world of distributed computing, Spark has simplified development and opened the doors for many to start writing distributed programs. Folks with little to no distributed coding experience can now write just a couple lines of code that will immediately get hundreds or thousands of machines working on creating business value.

Even though Spark code is easy to write and read, that doesn’t mean that users don’t run into issues of long-running, slow-performing jobs or out-of-memory errors. Thankfully most of the issues with using Spark have nothing to do with Spark but rather the approach taken when using it. Ted Malaska and Mark Grover cover the top five things that prevent Spark developers from getting the most out of their Spark clusters. When these issues are addressed, it is not uncommon to see the same job running 10x or 100x faster with the same clusters and the same data, using just a different approach.

Photo of Ted Malaska

Ted Malaska

Blizzard Entertainment

Ted Malaska is a group technical architect on the Battle.net team at Blizzard, helping support great titles like World of Warcraft, Overwatch, and HearthStone. Previously, Ted was a principal solutions architect at Cloudera helping clients find success with the Hadoop ecosystem and a lead architect at the Financial Industry Regulatory Authority (FINRA). He has also contributed code to Apache Flume, Apache Avro, Apache Yarn, Apache HDFS, Apache Spark, Apache Sqoop, and many more. Ted is a coauthor of Hadoop Application Architectures, a frequent speaker at many conferences, and a frequent blogger on data architectures.

Photo of Mark Grover

Mark Grover

Cloudera

Mark Grover is a software engineer working on Apache Spark at Cloudera. Mark is a committer on Apache Bigtop, a committer and PPMC member on Apache Spot (incubating) and a committer and PMC member on Apache Sentry and has contributed to a number of open source projects including Apache Hadoop, Apache Hive, Apache Sqoop, and Apache Flume. He is a coauthor of Hadoop Application Architectures and also wrote a section in Programming Hive. Mark is a sought-after speaker on topics related to big data at various national and international conference. He occasionally blogs on topics related to technology.

Comments on this page are now closed.

Comments

Picture of Mark Grover
Mark Grover
12/14/2016 9:55am SGT

Shikha, the slides are at tiny.cloudera.com/spark-mistakes

shikha tyagi
12/14/2016 2:00am SGT

Hi Mark, Can you share the slides used during session ?