Presented By O'Reilly and Cloudera
December 5-6, 2016: Training
December 6–8, 2016: Tutorials & Conference
Singapore

Storage designs done right equal faster processing and access

Ted Malaska (Blizzard)
4:15pm–4:55pm Wednesday, December 7, 2016
Hadoop use cases
Location: 321/322 Level: Intermediate
Average rating: *****
(5.00, 2 ratings)

Prerequisite Knowledge

  • Familiarity with data modeling

What you'll learn

  • Explore a set of optimal storage design patterns and learn the reasons for the resulting performance and speed of each

Description

The recent advancement in distributed processing engines, from Spark to Impala to Spark Streaming or Storm, has proved exciting. However, if your design only focuses on the processing layer to get speed and power then you may be missing half the story, leaving a significant amount of optimization untapped. Ted Malaska looks down the stack and describes a set of storage design patterns and schemas implemented on HBase, Kudu, Kafka, SolR, HDFS, and S3. By carefully tailoring how data is stored for each use case, processing and access times can be reduced by two to three orders of magnitude.

Photo of Ted Malaska

Ted Malaska

Blizzard

Ted Malaska is a senior solution architect at Blizzard. Previously, he was a principal solutions architect at Cloudera. Ted has 18 years of professional experience working for startups, the US government, some of the world’s largest banks, commercial firms, bio firms, retail firms, hardware appliance firms, and the largest nonprofit financial regulator in the US and has worked on close to one hundred clusters for over two dozen clients with over hundreds of use cases. He has architecture experience across topics including Hadoop, Web 2.0, mobile, SOA (ESB, BPM), and big data. Ted is a regular contributor to the Hadoop, HBase, and Spark projects, a regular committer to Flume, Avro, Pig, and YARN, and the coauthor of Hadoop Application Architectures.

Comments on this page are now closed.

Comments

01/12/2017 9:33pm SGT

Hi Ted,
I know this is a little late but are you able to share the slides to this talk?
Best regards,
Anton