Presented By O'Reilly and Cloudera
Make Data Work
Dec 4–5, 2017: Training
Dec 5–7, 2017: Tutorials & Conference
Singapore

FPGA-based acceleration architecture for Spark SQL

Xie Qi (Intel), Quanfu Wang (Intel China)
4:15pm4:55pm Thursday, December 7, 2017

Who is this presentation for?

  • Big data engineers and users

Prerequisite knowledge

  • A working knowledge of Spark SQL

What you'll learn

  • Explore a configurable FPGA-based Spark SQL acceleration architecture that leverages FPGAs' very high parallel computing capability to tremendously accelerate Spark SQL queries

Description

Spark SQL is one of the most widely used components in big data. Xie Qi and Quanfu Wang offer an overview of a configurable FPGA-based Spark SQL acceleration architecture that leverages FPGAs’ very high parallel computing capability to tremendously accelerate Spark SQL queries and FPGAs’ power efficiency to lower power consumption. Using SQL query decomposition algorithms, the architecture is able to decompose a complex SQL query into basic operations. According to their patterns, each is fed into an engine unit, which performs basic computation of sub string, arithmetic, and logic operations. Engine units are highly configurable and can be chained together to perform complex Spark SQL queries. Finally, a SQL query is transformed into a hardware pipeline.

Xie and Quanfu detail the performance benchmark results, which show that the FGPA-based Spark SQL acceleration architecture, run on Xeon E5s and FPGAs, offers a 10x–100x improvement, and demonstrate one SQL query workload from a real customer.

Photo of Xie Qi

Xie Qi

Intel

Xie Qi is a senior software engineer on the big data engineering team at Intel China, where he works on Spark optimization for Intel platforms. Xie has broad experience across big data, multimedia, and wireless.

Photo of Quanfu Wang

Quanfu Wang

Intel China

Quanfu Wang is a senior architect on Intel’s big data team, where he is working on software optimization and acceleration on information architecture and heterogeneous computing. Previously, Quanfu was a lead software engineer at Alcatel-Lucent, where he worked for the company’s wireline business group.