Presented By O'Reilly and Cloudera
Make Data Work
22–23 May 2017: Training
23–25 May 2017: Tutorials & Conference
London, UK

Speed up big data encryption in Apache Hadoop and Spark

Haifeng Chen (Intel)
14:0514:45 Thursday, 25 May 2017
Platform Security and Cybersecurity
Location: Capital Suite 14
Level: Intermediate
Average rating: ****.
(4.00, 2 ratings)

Who is this presentation for?

  • Security architects or engineers, big data architects or engineers, and product managers

Prerequisite knowledge

  • A basic understanding of distributed systems like HDFS and Spark, how data encryption works, and encryption algorithms

What you'll learn

  • Understand the problem and performance challenges of data encryption posed by the existing implementations
  • Learn proven ways to solve the challenges of various data encryption situations in Hadoop and Spark


As the processing capability of the modern platforms come to memory speed, securing big data using encryption usually hurts performance. Enterprises that choose performance over data security face serious risk of data leaks, causing increasing damage to their business in the big data era.

Haifeng Chen explores the data encryption problems and performance challenges posed by existing implementations, shares proven ways to speed up data encryption in Hadoop and Spark, as well as the latest progress in open source, and demystifies using hardware acceleration technology to protect your data.

Topics include:

  • The performance state and challenges from Java AES Cipher and SASL DIGEST-MD5-based RPC and network encryption
  • Why and how to utilize Apache Commons crypto to speed up encryption performance based on hardware acceleration
  • The latest work the community has done in Apache Hadoop and Spark to solve the performance challenges of encryption
Photo of Haifeng Chen

Haifeng Chen


Haifeng Chen is a senior software architect at Intel’s Asia Pacific R&D Center. He has more than 12 years’ experience in software design and development, big data, and security, with a particular interest in image processing. Haifeng is the author of image browsing, editing, and processing software ColorStorm.