Spark SQL is one of the most widely used components in big data. Xie Qi and Quanfu Wang offer an overview of a configurable FPGA-based Spark SQL acceleration architecture that leverages FPGAs’ very high parallel computing capability to tremendously accelerate Spark SQL queries and FPGAs’ power efficiency to lower power consumption. Using SQL query decomposition algorithms, the architecture is able to decompose a complex SQL query into basic operations. According to their patterns, each is fed into an engine unit, which performs basic computation of sub string, arithmetic, and logic operations. Engine units are highly configurable and can be chained together to perform complex Spark SQL queries. Finally, a SQL query is transformed into a hardware pipeline.
Xie and Quanfu detail the performance benchmark results, which show that the FGPA-based Spark SQL acceleration architecture, run on Xeon E5s and FPGAs, offers a 10x–100x improvement, and demonstrate one SQL query workload from a real customer.
Xie Qi is a senior software engineer on the big data engineering team at Intel China, where he works on Spark optimization for Intel platforms. Xie has broad experience across big data, multimedia, and wireless.
Quanfu Wang is a senior architect on Intel’s big data team, where he is working on software optimization and acceleration on information architecture and heterogeneous computing. Previously, Quanfu was a lead software engineer at Alcatel-Lucent, where he worked for the company’s wireline business group.
©2017, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com