Presented By O’Reilly and Cloudera
Make Data Work
21–22 May 2018: Training
22–24 May 2018: Tutorials & Conference
London, UK
See passes & pricing

Running data analytic workloads in the cloud

Mala Ramakrishnan (Cloudera), Eugene Fratkin (Cloudera), Mark Samson (Cloudera), Vinithra Varadharajan (Cloudera), Jason Wang (Cloudera)
9:00–12:30 Tuesday, 22 May 2018
Data engineering and architecture
 Location: Capital Suite 13 Level: Intermediate

Who is this presentation for?

  • Data engineers, developers, data scientists, system architects, system administrators, and those working in information security

Prerequisite knowledge

  • A basic understanding of data warehousing

Materials or downloads needed in advance

  • A WiFi-enabled laptop
  • To work with a command-line interface (optional), install Python 2.7 or above (Be able to install packages using PIP.)

What you'll learn

  • Learn how to create data pipelines and manage them in the cloud and in hybrid cloud environments
  • Understand how to implement metadata sharing and discovery across data applications

Description

Over the past several years, ever-increasing quantities of data are being processed within public clouds. The cloud promises to provide solutions to some of the limitations of conventional single multipurpose clusters offering hyperscale storage decoupled from elastic, on-demand compute and allows data to be shared between on-demand provisioned processing engines such as Hive, Spark, and Impala. But to fulfill this promise, you first need to solve several technical challenges: simple resource allocation, cross-cluster metadata sharing, and a common authorization framework. Without comprehensive answers to these questions, the challenges of single cluster model are simply duplicated inside a public cloud environment.

The cloud enables the delivery of solutions to single, multipurpose clusters offering hyperscale storage decoupled from elastic, on-demand computing. Mala Ramakrishnan, Eugene Fratkin, and Mark Samson detail new paradigms to effectively run production-level pipelines with minimal operational overhead. As a part of the deep dive, they also walk you through creating such a pipeline and executing data processing and data analytic workflows. Join in to learn how to remove barriers to data discovery, metadata sharing, and access control.

Photo of Mala Ramakrishnan

Mala Ramakrishnan

Cloudera

Mala Ramakrishnan heads product initiatives for Cloudera Altus – big data platform-as-a-service. She has 17+ years experience in product management, marketing, and software development in organizations of varied sizes that deliver middleware, software security, network optimization, and mobile computing. She holds a master’s degree in computer science from Stanford University.

Photo of Eugene Fratkin

Eugene Fratkin

Cloudera

Eugene Fratkin is a director of engineering at Cloudera, where he leads the company’s cloud infrastructure efforts. He was one of the founding members of the Apache MADlib project (scalable in-database algorithms for machine learning). Previously, Eugene was a cofounder of a Sequoia Capital-backed company focusing on applications of data analytics to problems of genomics. He holds PhD in computer science from Stanford University’s AI lab.

Photo of Mark Samson

Mark Samson

Cloudera

Mark Samson is a principal systems engineer at Cloudera, helping customers solve their big data problems using enterprise data hubs based on Hadoop. Mark has 17 years’ experience working with big data and information management software in technical sales, service delivery, and support roles.

Photo of Vinithra Varadharajan

Vinithra Varadharajan

Cloudera

Vinithra Varadharajan is an engineering manager in the cloud organization at Cloudera, where she is responsible for products such as Cloudera Director and Cloudera’s usage-based billing service. Previously, Vinithra was a software engineer at Cloudera, working on Cloudera Director and Cloudera Manager with a focus on automating Hadoop lifecycle management.

Photo of Jason Wang

Jason Wang

Cloudera

Jason is a software engineer at Cloudera focusing on the cloud.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)