Presented By O'Reilly and Cloudera
Make Data Work
September 25–26, 2017: Training
September 26–28, 2017: Tutorials & Conference
New York, NY

Julia and Spark, better together

Viral Shah (Julia Computing), Stefan Karpinski (The Julia Language)
2:05pm2:45pm Thursday, September 28, 2017
Spark & beyond
Location: 1A 08/10 Level: Intermediate

Who is this presentation for?

  • CTOs, data scientists, and architects

Prerequisite knowledge

  • A basic understanding of Spark or Julia

What you'll learn

  • Explore Julia and discover how it works with the Hadoop and Spark ecosystem


Spark is a fast and general engine for large-scale data. Julia is a fast and general engine for large-scale compute. Viral Shah and Stefan Karpinski explain how combining Julia’s compute and Spark’s data processing capabilities makes amazing things possible.

Viral and Stefan offer an overview of Julia, which solves the two language problem in multiple computational domains. Julia is simultaneously fast and easy and is quickly becoming the language of choice for data scientists, statisticians, quants, actuaries, chemists, physicists, biologists, psychologists, and all other applied mathematicians worldwide. With Spark.jl, a Julia package that makes it possible to run Julia in a Spark cluster and exchange data with Spark, these mathematicians now have a first-class method to access data from Spark in Julia and leverage Julia’s computational capabilities. Viral and Stefan also share a real-world example from one of the world’s largest insurers and the lessons learned along the way.

Photo of Viral Shah

Viral Shah

Julia Computing

Viral Shah is the cofounder and CEO of Julia Computing and a cocreator of the Julia language, as well as other open source software. Previously, he drove the rearchitecting of the government’s social security systems in India as part of the national ID project, Aadhaar. Viral is the coauthor of Rebooting India.

Photo of Stefan Karpinski

Stefan Karpinski

The Julia Language

Stefan Karpinski is one of the cocreators and core developers of the Julia language. He is an applied mathematician and data scientist by trade, having worked at Akamai, Citrix Online, and Etsy, but currently is focused on advancing Julia’s design, implementation, documentation, and community.