Sep 23–26, 2019

Reza Shiftehfar
Engineering Manager, Uber

Reza Shiftehfar leads the Hadoop platform teams at Uber, which help build and grow Uber’s reliable and scalable big data platform that serves petabytes of data utilizing technologies such as Apache Hadoop, Apache Hive, Apache Kafka, Apache Spark, and Presto. Reza is one of the founding engineers of Uber’s data team and helped scale Uber’s data platform from a few terabytes to over 100 petabytes while reducing the data latency from 24+ hours to minutes. Reza holds a PhD in computer science from the University of Illinois, Urbana-Champaign focused on building mobile hybrid cloud applications.

Sessions

2:05pm–2:45pm Thursday, September 26, 2019

Location: 1A 23/24

Creating an extensible 100+ PB real-time big data platform by unifying storage and serving

Data Engineering and Architecture

Secondary topics: Data Integration and Data Processing, Data Management and Storage, Data, Analytics, and AI Architecture, Transportation and Logistics

Reza Shiftehfar (Uber)

Building a reliable big data platform is extremely challenging when it has to store and serve hundreds of petabytes of data in real time. Reza Shiftehfar reflects on the challenges faced and proposes architectural solutions to scale a big data platform to ingest, store, and serve 100+ PB of data with minute-level latency while efficiently utilizing the hardware and meeting security needs. Read more.