Presented By O'Reilly and Cloudera
Make Data Work
Sept 29–Oct 1, 2015 • New York, NY

Building a production-ready data lake in the cloud

Prat Moghe (Cazena)
4:35pm–5:15pm Wednesday, 09/30/2015
Production Ready Hadoop
Location: 3D 05/08 Level: Intermediate
Average rating: ****.
(4.50, 2 ratings)
Slides:   1-PDF 

Hadoop’s ability to handle large amounts of varied data has been a critical enabler of the big data revolution. Many organizations’ ambitions to become more data-driven, however, are held back by a shortage of resources as well as the time and expense needed to purchase and set up Hadoop’s hardware and software underpinnings. The cloud offers a natural alternative that overcomes these barriers and also promises superior economics with elastic scaling.

In this session, attendees will learn the key steps and best practices for implementing Hadoop in the cloud, to support a production-ready data lake use case. The process will be broken down into clear steps, including:

  • Planning: understanding which workloads are a good match for Hadoop, selecting a distribution, performance benchmarking, scoping memory, and compute resource requirements
  • Provisioning: setting up public cloud resources and installing Hadoop
  • Moving data: connecting to data sources, encrypting data in movement, optimizing WAN transfer
  • Connecting and using analytical tools: ensuring a seamless transition for end-users for all workloads, monitoring and guaranteeing workload SLAs are met
  • Managing infrastructure: monitoring health, ensuring availability, disaster recovery
  • Ensuring security: data encryption, managing users/roles/groups, key management, VPN

The session will conclude by sharing additional lessons learned, best practices and business outcomes.

Photo of Prat Moghe

Prat Moghe

Cazena

Prat Moghe is the founder and CEO of Cazena. Prat is a successful big data entrepreneur with nearly 20 years of experience inventing next-generation products and building strong teams in the technology sector. Prior to founding Cazena, as SVP of strategy, products, and marketing at Netezza, Prat led a 400-person team that launched the latest-generation Netezza appliance, which led the market in price and performance. Netezza was acquired by IBM for $1.7B in 2010.