Presented By O'Reilly and Cloudera
Make Data Work
Sept 29–Oct 1, 2015 • New York, NY

Big data at Netflix: Faster and easier

Kurt Brown (Netflix)
11:20am–12:00pm Thursday, 10/01/2015
Data Innovations
Location: 1 E18 / 1 E19 Level: Intermediate
Tags: media, featured
Average rating: ****.
(4.87, 31 ratings)
Slides:   external link

The Netflix Data Platform is a constantly evolving, large-scale infrastructure running in the (AWS) cloud. This talk will dive into what we’re up to and why. We are especially focused on performance and ease of use. We’ve upgraded to Hadoop 2, have partnered with the community developing Pig on Tez, have adopted the Parquet file format, and fully integrated Presto into our stack. We are exploring Spark for streaming, machine learning, and analytic use cases.

We continue to add to our big data, open source suite, with our latest contribution Inviso (which provides easy searching and visibility into Hadoop execution and performance). We are also heads-down developing a cohesive framework for easy platform interaction (via our big data API and big data portal). We’ll talk through these technologies and how they are benefiting the Netflix business. We’ll also dive into how we do things differently at Netflix (vs. most other companies), notably the motivations behind our architecture/ approach and the benefits that we (and hopefully you can) achieve.

Photo of Kurt Brown

Kurt Brown

Netflix

Kurt Brown leads the data platform team at Netflix, which architects and manages the technical infrastructure underpinning the company’s analytics, including various big data technologies like Hadoop, Spark, and Presto, machine learning infrastructure for Netflix data scientists, and traditional BI tools including Tableau.