Presented By
O’Reilly + Cloudera
Make Data Work
March 25-28, 2019
San Francisco, CA
Please log in

Cost-effective Presto on AWS with Spot nodes

Shubham Tagra (Qubole)
11:00am–11:40am Wednesday, March 27, 2019
Data Engineering & Architecture
 Location: 2004
Secondary topics:  AI and Data technologies in the cloud
Average rating: ***..
(3.50, 8 ratings)
Download slides (PPTX)

Who is this presentation for?

  • Presto users, developers, and database administrators

Level

Intermediate

Prerequisite knowledge

  • Familiarity with PrestoDB

What you'll learn

  • Learn how to run Presto workloads on AWS Spot nodes reliably
  • Understand the finer details of Presto that limit usage of Spot nodes and the subtleties of the query retry system

Description

Presto is gaining popularity for interactive and reporting use cases owing to its performance and ease of use. Presto is performant because it executes pipelined in-memory query plans, but its default architecture isn’t amenable to handling Spot node loss. However, many users still use Spot nodes for their cost benefits and tolerate query failures as a necessary evil.

With Spot termination notification listeners and its query retry system, Presto improves the user experience. The changes also allow users who cannot tolerate failures to use Spot nodes and enjoy their cost benefits.

Shubham Tagra explores the gaps in Presto architecture, explains how to use Spot nodes, covers enhancements like Spot termination notification listeners and the query retry system, and showcases the improvements in terms of reliability and TCO achieved through them.

Photo of Shubham Tagra

Shubham Tagra

Qubole

Shubham Tagra is a senior staff engineer at Qubole, working on Presto and Hive development and making these solutions cloud ready. Previously, Shubham worked on the storage area network at NetApp. Shubham holds a bachelor’s degree in computer engineering from the National Institute of Technology, Karnataka, India.