Presented By O'Reilly and Cloudera
Make Data Work
September 25–26, 2017: Training
September 26–28, 2017: Tutorials & Conference
New York, NY

How to successfully run data pipelines in the cloud

Jennifer Wu (Cloudera), Philip Langdale (Cloudera), Kostas Sakellis (Cloudera)
2:55pm3:35pm Thursday, September 28, 2017
Big data and the Cloud, Data Engineering & Architecture
Location: 1A 15/16/17 Level: Intermediate
Secondary topics:  Architecture

Who is this presentation for?

  • Data engineers, ETL developers, and Hadoop administrators

What you'll learn

  • Understand the price and performance impact of different deployment models for data engineering clusters and the advantages of data engineering delivered as a service
  • Explore the latest cloud technologies and learn how they can be leveraged by data engineers for an optimal end-user experience

Description

With its scalable data store, elastic compute, and pay-as-you-go cost model, cloud infrastructure is well-suited for large-scale data engineering workloads, especially those such as ETL and model training batch workloads that use Hive and Spark compute engines. Jennifer Wu, Philip Langdale, and Kostas Sakellis explain how data engineers can leverage the cloud in order to successfully run data engineering workloads. They explore the latest cloud technologies, focusing on data engineering workloads, cost, security, and ease-of-use implications for data engineers, and cover the advantages of the managed service deployment model and security best practices to demonstrate how to apply these technologies in your own projects.

Photo of Jennifer Wu

Jennifer Wu

Cloudera

Jennifer Wu is director of product management for cloud at Cloudera, where she focuses on cloud strategy and solutions. Previously, Jennifer worked as a product line manager at VMware, working on the vSphere and Photon system management platforms.

Photo of Philip Langdale

Philip Langdale

Cloudera

Philip Langdale is the engineering lead for cloud at Cloudera. He joined the company as one of the first engineers building Cloudera Manager and served as an engineering lead for that project until moving to working on cloud products. Previously, Philip worked at VMware, developing various desktop virtualization technologies. Philip holds a bachelor’s degree with honors in electrical engineering from the University of Texas at Austin.

Kostas Sakellis

Cloudera

Kostas Sakellis is the lead and engineering manager of the Apache Spark team at Cloudera. Kostas holds a bachelor’s degree in computer science from the University of Waterloo, Canada.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)