Presented By O'Reilly and Cloudera
Make Data Work
Dec 4–5, 2017: Training
Dec 5–7, 2017: Tutorials & Conference
Singapore

How to successfully run data pipelines in the cloud

Kostas Sakellis (Cloudera), Philip Langdale (Cloudera)
12:05pm12:45pm Wednesday, December 6, 2017
Location: 310/311

Who is this presentation for?

  • Data engineers, ETL developers, and Hadoop administrators

What you'll learn

  • Understand the price and performance impact of different deployment models for data engineering clusters and the advantages of data engineering delivered as a service
  • Explore the latest cloud technologies and learn how they can be leveraged by data engineers for an optimal end-user experience

Description

With its scalable data store, elastic compute, and pay-as-you-go cost model, cloud infrastructure is well-suited for large-scale data engineering workloads, especially those such as ETL and model training batch workloads that use Hive and Spark compute engines. Kostas Sakellis explains how data engineers can leverage the cloud in order to successfully run data engineering workloads. Kostas explores the latest cloud technologies, focusing on data engineering workloads, cost, security, and ease-of-use implications for data engineers, and covers the advantages of the managed service deployment model and security best practices to demonstrate how to apply these technologies in your own projects.

Kostas Sakellis

Cloudera

Kostas Sakellis is the lead and engineering manager of the Apache Spark team at Cloudera. Kostas holds a bachelor’s degree in computer science from the University of Waterloo, Canada.

Photo of Philip Langdale

Philip Langdale

Cloudera

Philip Langdale is the engineering lead for cloud at Cloudera. He joined the company as one of the first engineers building Cloudera Manager and served as an engineering lead for that project until moving to working on cloud products. Previously, Philip worked at VMware, developing various desktop virtualization technologies. Philip holds a bachelor’s degree with honors in electrical engineering from the University of Texas at Austin.