Presented By O'Reilly and Cloudera
December 5-6, 2016: Training
December 6–8, 2016: Tutorials & Conference
Singapore

Top five mistakes when writing Spark applications

Ted Malaska (Blizzard), Mark Grover (Cloudera)
11:15am–11:55am Wednesday, December 7, 2016
Spark & beyond
Location: Summit 2 Level: Intermediate
Average rating: ****.
(4.12, 8 ratings)

Prerequisite Knowledge

  • A working knowledge of Spark

What you'll learn

  • Understand common mistakes and how to fix them, in order to write more capable, powerful Spark applications

Description

In the world of distributed computing, Spark has simplified development and opened the doors for many to start writing distributed programs. Folks with little to no distributed coding experience can now write just a couple lines of code that will immediately get hundreds or thousands of machines working on creating business value.

Even though Spark code is easy to write and read, that doesn’t mean that users don’t run into issues of long-running, slow-performing jobs or out-of-memory errors. Thankfully most of the issues with using Spark have nothing to do with Spark but rather the approach taken when using it. Ted Malaska and Mark Grover cover the top five things that prevent Spark developers from getting the most out of their Spark clusters. When these issues are addressed, it is not uncommon to see the same job running 10x or 100x faster with the same clusters and the same data, using just a different approach.

Photo of Ted Malaska

Ted Malaska

Blizzard

Ted Malaska is a senior solution architect at Blizzard. Previously, he was a principal solutions architect at Cloudera. Ted has 18 years of professional experience working for startups, the US government, some of the world’s largest banks, commercial firms, bio firms, retail firms, hardware appliance firms, and the largest nonprofit financial regulator in the US and has worked on close to one hundred clusters for over two dozen clients with over hundreds of use cases. He has architecture experience across topics including Hadoop, Web 2.0, mobile, SOA (ESB, BPM), and big data. Ted is a regular contributor to the Hadoop, HBase, and Spark projects, a regular committer to Flume, Avro, Pig, and YARN, and the coauthor of O’Reilly Media’s Hadoop Application Architectures.

Photo of Mark Grover

Mark Grover

Cloudera

Mark Grover is a software engineer working on Apache Spark at Cloudera. Mark is a committer on Apache Bigtop and a committer and PMC member on Apache Sentry and has contributed to a number of open source projects including Apache Hadoop, Apache Hive, Apache Sqoop, and Apache Flume. He is a coauthor of Hadoop Application Architectures and also wrote a section in Programming Hive. Mark is a sought-after speaker on topics related to big data at various national and international conference. He occasionally blogs on topics related to technology.

Comments on this page are now closed.

Comments

Picture of Mark Grover
Mark Grover
12/14/2016 9:55am SGT

Shikha, the slides are at tiny.cloudera.com/spark-mistakes

shikha tyagi
12/14/2016 2:00am SGT

Hi Mark, Can you share the slides used during session ?