Presented By O'Reilly and Cloudera
Make Data Work
Feb 17–20, 2015 • San Jose, CA

New Directions for Spark in 2015

Matei Zaharia (Databricks)
9:25am–9:35am Friday, 02/20/2015
Location: Grand Ballroom 220
Average rating: ****.
(4.11, 37 ratings)

As the Apache Spark userbase grows, the developer community is working to adapt it for ever-wider use cases. 2014 saw fast adoption of Spark in the enterprise and major improvements in its performance, scalability and standard libraries. In 2015, we want to make Spark accessible to a wider set of users, through new high-level APIs for data science: machine learning pipelines, data frames, and R language bindings. In addition, we are defining extension points to let Spark grow as a platform, making it easy to plug in data sources, algorithms, and external packages. Like all work on Spark, these APIs are designed to plug seamlessly into Spark applications, giving users a unified platform for streaming, batch and interactive data processing.

Photo of Matei Zaharia

Matei Zaharia


Matei Zaharia started the Spark project at UC Berkeley and is currently CTO of Databricks. He serves as Spark’s vice president at Apache. In spring 2015, he is also beginning an assistant professor position at MIT.