Presented By O'Reilly and Cloudera
Make Data Work
September 26–27, 2016: Training
September 27–29, 2016: Tutorials & Conference
New York, NY

Architecting a data platform

John Akred (Silicon Valley Data Science), Stephen O'Sullivan (Data Whisperers), Mauricio Vacas (Silicon Valley Data Science)
9:00am–12:30pm Tuesday, 09/27/2016
Spark & beyond
Location: Hall 1C Level: Intermediate
Average rating: ***..
(3.07, 15 ratings)

Prerequisite knowledge

  • A basic knowledge of Hadoop and Spark
  • Materials or downloads needed in advance

  • A laptop (You'll be provided a GitHub link for sample code.)
  • What you'll learn

  • Understand how the various parts of the Hadoop and big data ecosystem fit together in production to create a data platform supporting batch, interactive, and real-time analytical workloads
  • Description

    What are the essential components of a data platform? John Akred, Mauricio Vacas, and Stephen O’Sullivan explain how the various parts of the Hadoop, Spark, and big data ecosystems fit together in production to create a data platform supporting batch, interactive, and real-time analytical workloads.

    By tracing the flow of data from source to output, John, Mauricio, and Stephen explore the options and considerations for components, including acquisition from internal and external data sources; ingestion (offline and real-time processing); storage; analytics (batch and interactive); and providing data services (exposing data to applications). They’ll also give advice on tool selection, the function of the major Hadoop components and other big data technologies such as Spark and Kafka, and integration with legacy systems.

    Photo of John Akred

    John Akred

    Silicon Valley Data Science

    With over 15 years in advanced analytical applications and architecture, John Akred is dedicated to helping organizations become more data driven. As CTO of Silicon Valley Data Science, John combines deep expertise in analytics and data science with business acumen and dynamic engineering leadership.

    Photo of Stephen O'Sullivan

    Stephen O'Sullivan

    Data Whisperers

    A leading expert on big data architectures, Stephen O’Sullivan has 25 years of experience creating scalable, high-availability data and applications solutions. A veteran of Silicon Valley Data Science, @WalmartLabs, Sun, and Yahoo. Stephen is an independent adviser to enterprises on all things data..

    Photo of Mauricio Vacas

    Mauricio Vacas

    Silicon Valley Data Science

    Mauricio Vacas is a data engineer at Silicon Valley Data Science, where he has developed in multiple areas of the data platform from ingestion to analysis and visualization. Previously, Mauricio was a tech arch manager with 5+ years of experience working in Accenture’s R&D group and its big data practice, where he managed the development of a cloud-based data platform used by Accenture’s data science teams to create analytic models for multiple customer projects. Mauricio is passionate about technology and its ability to make a difference in people’s lives.

    Comments on this page are now closed.


    Picture of Mauricio Vacas
    Mauricio Vacas
    09/27/2016 10:24am EDT

    Hello everyone. Thanks for attending. If you’re interested in the slides, you can request them from this URL:


    Picture of Daniel Wiest
    09/27/2016 10:19am EDT

    Can you please provide the link for the download?

    09/27/2016 7:23am EDT

    Hi can you please send us a link for the presentation

    09/26/2016 6:39am EDT

    Hi, Thanks for presenting a data platform architecture session. Could you please include Data Management/Data Governance on HDFS also in this session? If it is too big to include, you can provide an overview and guide us to additional resources.

    09/23/2016 10:06am EDT

    Is there any prerequisite needed for the hands-on stuff other than bringing the laptop

    08/31/2016 5:44am EDT

    Could you please provide some description of the hands-on portion of this? How will code help us learn architecture? How critical is participation in that?