Volunteers around the world increasingly act as human sensors to collect millions of data points. A team from the World Bank recently trained deep learning models to confirm that photos gathered through a crowdsourced data collection pilot matched the goods for which observations were submitted, using Apache Spark and BigDL—a distributed deep learning library designed from the ground up to run natively on Apache Spark that enables data engineers and scientists to write deep learning applications in Scala or Python as standard Spark programs, without having to explicitly manage distributed computations.
Yulia Tell and Maurice Nsabimana walk you through getting started with BigDL—which runs in any Apache Spark environment, whether on-premises or in the cloud—and explain how to write a deep learning application that leverages Spark to train image recognition models at scale. Along the way, Yulia and Maurice detail a collaborative project to design and train large-scale deep learning models using crowdsourced images from around the world.
Yulia Tell is a technical program manager on the big data technologies team within the Software and Services Group at Intel, where she is working on several open source projects and partner engagements in the big data domain. Yulia’s work is focused specifically on Apache Hadoop and Apache Spark, including big data analytics applications that use machine learning and deep learning. Yulia holds an MSc in computer science from Moscow Power Engineering Technical University and has completed executive training on market driving strategies at London Business School.
Maurice Nsabimana is a statistician focusing on national accounts and macroeconomic indicators in the World Bank’s Development Data Group. Previously, Maurice worked in the private sector and civil society and at a think tank. His research interests lie at the intersection of computational economics, machine learning, and public policy and in the development of new, practical methods and information technologies that can be directly applied to strengthen local capacity. He holds an MA in international affairs from the School of International and Public Affairs at Columbia University and a BSc in computer science from Vesalius College in Brussels, Belgium.
©2018, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com