Mar 15–18, 2020

Usability First: the Evolution of Robinhood’s Data Platform

Shuo Xiang (Robinhood), Grace Lu (Robinhood), Peng Liu (Robinhood), Arpan Shah (Robinhood)

Who is this presentation for?

Data engineers, data architects, developers

Level

Beginner

Description

The Data Platform at Robinhood has evolved considerably as the scale of our data and needs of the company have evolved. In this talk, we will introduce the pain points and challenges that motivated the development of our first data-lake. We will also discuss how we evolved this into a platform with a focus on usability as the driving force. We will share learnings from balancing moving fast to delivering a valuable tool and platform. Finally we will showcase our journey from using cloud-based tools to a self-hosted computational framework in our endeavor to keep increasing productivity for our customers.

Topics include:
– Common and special data workflows in Robinhood
– How do we start fast with minimum resources while putting usability first
– The architecture of Robinhood’s batch, interactive and streaming computation
– Operational toolset and processes to run the platform

Prerequisite knowledge

Basic understanding about common distributed computing systems (Hadoop/Spark/Hive/Presto etc) and analytical tools

What you'll learn

The lessons that Robinhoodies learned from the evolution of their data platform.
Photo of Shuo Xiang

Shuo Xiang

Robinhood

Shuo is a software engineer from Robinhood’s Data Platform team.

Photo of Grace Lu

Grace Lu

Robinhood

Grace Lu is a software engineer of Robinhood’s Data Platform team.

Photo of Peng Liu

Peng Liu

Robinhood

Peng Liu is a software engineer of Robinhood’s Data Platform team.

Photo of Arpan Shah

Arpan Shah

Robinhood

Arpan Shah is the engineering manager of Robinhood’s Data Engineering team.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

pr@oreilly.com

For media/analyst press inquires