Presented By O'Reilly and Cloudera
Make Data Work
Feb 17–20, 2015 • San Jose, CA

Designing Data Architectures for Robust Decision Making

Gwen Shapira (Confluent)
10:40am–11:20am Friday, 02/20/2015
Hadoop in Action
Location: 210 B/F
Average rating: ****.
(4.43, 7 ratings)

Organizations do not store, process and analyze data for their amusement. They plan to use the data to drive business decisions. If data validity is uncertain, the data is useless for decision making. In this session we will show how to design architectures that allow to prove and improve data validity at every step of the decision making process.

We will look at how to design a robust data processing architecture using Apache Hadoop and related tools and frameworks. The architectures we’ll discuss retain critical metadata that allows the business to later validate the data sources, data processing steps and data analysis when using the data to make decisions. Robust architectures make the cleanup, processing and analysis repeatable, so the process can be improved and data analysts can experiment with new analysis methods.

Photo of Gwen Shapira

Gwen Shapira


Gwen Shapira is a Solutions Architect at Cloudera and leader of IOUG Big Data SIG. Gwen Shapira studied computer science, statistics and operations research at the University of Tel Aviv, and then went on to spend the next 15 years in different technical positions in the IT industry. She specializes in scalable and resilient solutions and helps her customers build high-performance large-scale data architectures using Hadoop. Gwen Shapira is a frequent presenter at conferences and regularly publishes articles in technical magazines and her blog.