Presented By
O’Reilly + Cloudera
Make Data Work
29 April–2 May 2019
London, UK

How do you evolve your data infrastructure?

Neelesh Salian (Stitch Fix)
16:3517:15 Wednesday, 1 May 2019
Data Engineering and Architecture
Location: Capital Suite 8/9
Secondary topics:  Data Platforms, Data preparation, data governance, and data lineage, Retail and e-commerce
Average rating: ****.
(4.67, 3 ratings)

Who is this presentation for?

  • Data scientists, data engineers, big data developers, software engineers

Level

Intermediate

Prerequisite knowledge

  • Familiarity with data infrastructure (compute, execution, etc.), cloud infrastructure, and big data systems

What you'll learn

  • Learn how Stitch Fix built and sustains its data infrastructure

Description

Stitch Fix has come a long way, both as a company and as a data science–heavy team. Its business challenges its data teams in terms of scale and complexity.

Neelesh Salian discusses how Stitch Fix’s data platform team maintains and innovates its infrastructure for the company’s data scientists and evolves the ecosystem as the business continues to expand.

The team’s charter has always remained to build a self-service data platform for data scientists, empowering them to be full stack and responsible for their own workflows. To accomplish this goal, the team prioritizes impactful changes, invests time in prototyping and testing, and uses its own infrastructure to test and innovate. This enables the team to be autonomous as developers of data infrastructure while also focusing on the larger team mission.

The team takes a microservice architecture–based approach to solving critical problems. Neelesh shares details about such approaches the team took to change the ecosystem: adapting for data lineage, changing the reading and writing interfaces to the data warehouse, and improving the execution of Spark workflows, for instance. These were larger cross-functional projects that required careful planning and execution.

Join in to explore lessons the team learned during this evolution that have helped them grow and make better decisions for the future.

Photo of Neelesh Salian

Neelesh Salian

Stitch Fix

Neelesh Srinivas Salian is a software engineer on the data platform team at Stitch Fix, where he works on the compute infrastructure used by the company’s data scientists. Previously, he worked at Cloudera, where he worked with Apache projects like YARN, Spark, and Kafka.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)