Presented By O'Reilly and Cloudera
Make Data Work
31 May–1 June 2016: Training
1 June–3 June 2016: Conference
London, UK

Data science at scale: Using Spark and Hadoop

Kai Voigt (Cloudera)
9:00–17:00 Tuesday, 31/05/2016 - Wednesday, 01/06/2016
Location: Capital Suite 7

All training courses take place 9:00 - 17:00, Tuesday and Wednesday. In order to maintain a high level of hands-on learning and instructor interaction, each training is limited in size.

Participants should plan to attend both days of this 2-day training. Training passes do not include access to tutorials on Wednesday.

Average rating: ***..
(3.25, 4 ratings)

Prerequisite knowledge

This course is intended for two audiences: advanced analysts without distributed computing skills and engineers proficient in distributed computing without advanced analytical skills. Students should be comfortable with the Linux command line and have proficiency in a scripting language; Python is strongly preferred, but familiarity with Perl or Ruby is sufficient.

As an attendee of the session, you don’t have to prepare anything in advance. You will get access to all documents and software during the first morning. The training will be a lecture with demos, but attendees will be able to repeat all exercises themselves with the provided material.


Data scientists build information platforms to provide deep insight and answer previously unimaginable questions. Spark and Hadoop are transforming how data scientists work by allowing interactive and iterative data analysis at scale. Learn how Spark and Hadoop enable data scientists to help companies reduce costs, increase profits, improve products, retain customers, and identify new opportunities.

Kai Voigt explores what data scientists do, the problems they solve, and the tools and techniques they use. Through in-class simulations and exercises, Kai walks attendees through applying data science methods to real-world challenges in different industries, offering preparation for and experience of data scientist roles in the field.

Topics include:

  • How to identify potential business use cases where data science can provide impactful results
  • How to obtain, clean, and combine disparate data sources to create a coherent picture for analysis
  • What statistical methods to leverage for data exploration that will provide critical insight into your data
  • Where and when to leverage Hadoop streaming and Apache Spark for data science pipelines
  • What machine-learning technique to use for a particular data science project
  • How to implement and manage recommenders using Spark’s MLlib and how to set up and evaluate data experiments
  • The pitfalls of deploying new analytics projects to production at scale

Day 1:

  • Introduction
  • Data science overview and use cases
  • Project lifecycle
  • Data acquisition
  • Evaluating input data
  • Data transformation
  • Data analysis and statistical methods

Day 2:

  • Fundamentals of machine learning
  • Recommender overview
  • Introduction to Apache Spark and MLlib
  • Implementing recommenders with MLlib
  • Experimentation and evaluation
  • Production deployment and beyond
Photo of Kai Voigt

Kai Voigt


Kai Voigt is a senior instructor for Hadoop classes at Cloudera, delivering training classes for developers and administrators worldwide. Kai held the same role at MySQL, Sun, and Oracle. He has spoken at a number of O’Reilly conferences.

Comments on this page are now closed.


Picture of Kai Voigt
Kai Voigt
1/06/2016 19:55 BST

Thanks everyone for attending!

Picture of Abdurrahman Adebiyi
Abdurrahman Adebiyi
1/06/2016 15:45 BST

Large Scale data using Spark- and Hadoop- Clusters should have been used rather than small dataset of 5000-7500 . My expectation from the title was Data Science at (large)scale.

Picture of Sophia DeMartini
Sophia DeMartini
31/05/2016 10:29 BST

Hi Reni,

As Kai said, the trainings are not recorded, unfortunately. (We will be recording all keynotes, tutorials, and sessions, pending speaker consent).

Thank you,

Picture of Kai Voigt
Kai Voigt
29/05/2016 21:14 BST

Hi Reni.

no, the sessions won’t be recorded.


Picture of Abdurrahman Adebiyi
Abdurrahman Adebiyi
29/05/2016 21:13 BST

@Sophia Thank you very much for your response.

Reni Mo
29/05/2016 9:17 BST

Hi Sophia,
Missed the opportunity to attended. Will there be any video recordings of the sessions available later.

Lucy Langdon
27/05/2016 9:40 BST

Many thanks Sophia.

Picture of Sophia DeMartini
Sophia DeMartini
26/05/2016 20:14 BST

Hi Lucy,

There is not a waiting list, but I’ll hang on to your info, and if a spot opens up, I’ll contact you to let you know.

Thank you,

Lucy Langdon
26/05/2016 16:18 BST

Hi, could you let me know if there is a reserve list for attendees as I’d really like to attend but it’s already sold out.

Picture of Kai Voigt
Kai Voigt
25/05/2016 8:56 BST

Daniele, all files required will be downloadable during the class. So you don’t have to prepare anything in advance.

See you on Tuesday

Daniele Bonacorsi
25/05/2016 8:44 BST

Hi, I got a message to check the download needed to get prepared in advance. May you be so kind to point me to a download link? I am not finding any. Thanks!

Picture of Sophia DeMartini
Sophia DeMartini
20/05/2016 21:30 BST

Hi Abdurrahman – this is a two-day training, which will take place Tuesday, May 31st, and Wednesday, June 1st. Please let us know if you have any other questions.

Thank you,

Picture of Abdurrahman Adebiyi
Abdurrahman Adebiyi
20/05/2016 8:00 BST

Hello. Please, is this a 1-day training session.?