Presented By O'Reilly and Cloudera
Make Data Work
5–7 May, 2015 • London, UK

A taste of random decision forests on Apache Spark

Sean Owen (Cloudera)
10:55–11:35 Thursday, 7/05/2015
Data Science
Location: King's Suite - Balmoral
Average rating: ****.
(4.94, 17 ratings)

Prerequisite Knowledge

No knowledge of Spark or machine learning is required, although a basic familiarity with both will help.


Apache Spark continues to gain momentum as the new processing paradigm for Apache Hadoop, and for the data scientist, it has a lot to like: natively distributed, REPL, Python APIs in addition to native Scala, and a library of machine learning algorithms, MLlib.

Spark 1.2 includes an implementation of random decision forests, an important and popular ensemble classifier/regressor algorithm. This talk will introduce Spark, Scala, and random decision forests to the curious, and demonstrate the process of analyzing a real-world data set with them. The session will cover loading data and understanding the data set, and introduce ideas like training and test set evaluation, ensemble methods, feature types, and supporting concepts like impurity and entropy.

Attendees will:
- Become familiar with Spark basics using its Scala API
- Understand the decision tree and random decision forest algorithms
- See a simple, narrated data science workflow in action on a real data set

Photo of Sean Owen

Sean Owen


Sean is director of data science for EMEA at Cloudera. Previously, Sean founded Myrrix Ltd, producing a real-time recommender and clustering product evolved from Mahout. Myrrix is now part of Cloudera. Sean was a primary author of recommender components in Apache Mahout, and has been a committer and PMC member for the project. He is co-author of Advanced Analytics on Spark and Mahout in Action. Sean was previously a senior engineer at Google.