Helping the World’s Farmers Adapt to Climate Change

Hadoop: Case Studies, Beekman / Sutton North (NY Hilton)
Average rating: ***..
(3.50, 2 ratings)

Big Data takes on the planet’s toughest challenge: analyzing the weather’s complex and multi-layered behavior to help the world’s farmers adapt to climate change. Increased volatility in weather – the source of over 90% of crop loss – has in recent years become a major concern for farmers and our food supply. By combining modern Big Data techniques, climatology and agronomics, The Climate Corporation protects the $3 trillion global agriculture industry with automated hyper-local weather insurance.

TCC’s algorithms analyze millions of weather measurements, billions of soil observations, and trillions of simulation datapoints to quantify weather risk and price insurance policies carefully tailored to specific situations. Their systems, using Hadoop and tools built in the Clojure functional programming language, use thousands of cloud servers to process historical and forecast data and generate 10,000 weather scenarios, going out several years, in a dense grid covering the US. The resulting trillions of scenario datapoints—amounting to hundreds of terabytes—are used to quantify risk to crop yield and build corresponding insurance policies. Weather-related data is acquired multiple times a day directly from major climate models and incorporated into a real-time pricing engine. The quantity of data processed has grown an average of 10x every year as the company adds more granular geographic data, requiring highly scalable processes for rapid data acquisition and ingestion.

In this talk, CTO and company co-founder Siraj Khaliq will discuss the problem space, evolution of the business and corresponding technology, and how the company’s team of mathematicians and computer scientists is using state-of-the-art big data techniques today to tackle this very real-world problem.

Photo of Siraj Khaliq

Siraj Khaliq

The Climate Corporation

Siraj founded The Climate Corporation (formerly WeatherBill) in 2006, having previously worked at Google in multiple technical lead roles, from the company’s distributed computing infrastructure to the high-profile Google Book Search project and other offline content search initiatives. Siraj obtained an M.S. degree in Computer Science from Stanford University, and a B.A. (Hons.) in Computer Science from the University of Cambridge, England. While at Stanford, he was also a lead software architect for the popular Folding@Home distributed computing project.

Sponsors

Sponsorship Opportunities

For information on exhibition and sponsorship opportunities, contact Susan Stewart at sstewart@oreilly.com.

Media Partner Opportunities

For information on trade opportunities contact Kathy Yu at mediapartners
@oreilly.com

Press and Media

For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com

Contact Us

View a complete list of Strata contacts.