Doing Big Data All By Yourself: Interactive Data Driven Decision Making by Non-Programmers

Ari Gesher (Kairos Aerospace), Lauren Chaparro (Palantir Technologies)
Data and Analytics
Location: Plaza Room A Level: Intermediate
Average rating: *****
(5.00, 2 ratings)

As healthcare organizations gain access to more and more data, they rely increasingly on complex systems and predictive tools to make sense of it all. This has led to a divide between those who perform data analysis (programmers, statisticians, database administrators) and those who make decisions (physicians, insurance administrators, policy makers). In addition, the needs of those users are fairly heterogeneous, even a on a single data set.

In this presentation, we will show a working system that bridges the gap between data analysis and decision making using a carefully composed set of big-data technologies mated with an interactive, high-level interface. By leveraging a powerful backend infrastructure and an intuitive set of analytical tools, health professionals ranging from physicians to insurance analysts can perform interactive, real-time analysis on all data within their enterprise.

To build the system for this realtime, interactive demonstration, we integrated ten years of Medicare claims, including information from 700,000 physicians, 30 million beneficiaries, 100 million claims, and 1 billion medical procedures. We also integrated 20 million PubMed journal articles and information on one million physicians and 3000 hospitals using the National Provider Identifier Database and Medicare Hospital Compare.

The first analysis is from the perspective of a policy analyst who wants to better characterize Medicare’s “high spenders” — beneficiaries who rank among the top 0.01% of in terms of health spending – to drive policy decisions. We drill down on 30 million beneficiaries in seconds to identify high spenders and look at their providers, diagnoses, and procedures in more detail.

The second analysis is from the perspective of a single physician who wants guidance on a patient consult. In this analysis, the physician is able to determine the estimated total cost of this disease, the clusters of treatment strategies, and information detailing how much a set of treatments will cost and how painful they will be for the patient. The physician can also use the platform to zoom out for a summary of all of the physician’s patients and all patients at her hospital.

Our presentation underscores the importance of data-driven decision making, an endeavor that becomes more challenging as more data become available to health organizations. Most importantly, using the right composition of technologies and proper data integration, it’s possible to build a system that is flexible and extensible, letting the different types of users who are interested in health care data quickly answer nuanced questions that are rigorously backed by data.

Photo of Ari Gesher

Ari Gesher

Kairos Aerospace

Ari Gesher is the founding director of software engineering at Kairos Aerospace, a startup building and operating the next-generation of airborne and spaceborne sensors for monitoring oil and gas infrastructure. Ari also serves as consulting architect for Jupiter, a company productizing high-quality datasets that describe the long-term effects of climate change. Previously, he was a very early engineer at Palantir Technologies and later served as Palantir’s engineering ambassador to the tech community at large; before Palantir, he was the maintainer of the open source archive. Ari is the coauthor of The Architecture of Privacy, which explains how to responsibly hold data about people while preserving their privacy to the greatest extent possible. Ari is a frequent speaker on various topics, including the need for modern, high-leverage engineers to work on substantive problems, human-computer symbiosis as system design aesthetic, the limits of automated decision making, and privacy architectures for a world where everything is recorded.

Photo of Lauren Chaparro

Lauren Chaparro

Palantir Technologies

Lauren Chaparro is an Engineer and leads Palantir Health business development. Before joining Palantir, Lauren was a Life Sciences investor at Health Evolution Partners, a growth-stage private equity fund led by Dr. David Brailer. Previously, she was as a consultant at the Parthenon Group, focused on the healthcare and private equity practices. Prior to Parthenon, Lauren worked at Google, Inc., including membership on the Google Health team.

Lauren graduated cum laude from Princeton University, majoring in
Molecular Biology and the Woodrow Wilson School of Public and
International Affairs. She received her MBA from the Stanford Graduate School of Business.


For information on exhibition and sponsorship opportunities at the conference, contact Sharon Pierce at (203) 304-9476 or

For information on trade opportunities with O'Reilly conferences contact mediapartners

For media-related inquiries, contact Maureen Jennings at

View a complete list of Strata Rx contacts