Integrated Business Intelligence Suite: How Uber built a platform to convert raw data into knowledge

Description

Uber is changing the way people think about transportation. As an integral part of its business in 600+ cities across 65 countries, teams across Uber depend on data to power every decision. As a result, business intelligence is key to Uber’s future to help employees make real-time decisions at a global scale while adapting to unique, local conditions.

Not surprisingly, Uber heavily invests in business intelligence. But the people working with this data have a variety of skill levels. Folks with advanced skills know exactly what to do, but for the less experienced, it’s more iterative. They experiment with the data and try a few techniques to extract useful insights. Uber has multiple tools that help users drive data insights fast at various skill levels and works to ensure there’s a seamless integration across products that turns them into a suite rather than various siloed tools.

Uber built the Business Intelligence Suite (BI Suite) to navigate through hundreds of petabytes of data and millions of weekly queries, with the vision of democratizing data for users of all backgrounds—working on common data sources, getting real-time and batch metrics, having access to common resources, sharing insights easily and freely—without having to reinvent the wheel.

Visualization is also a key component of the toolset. Many of Uber’s user’s problems are spatiotemporal in nature. Two months ago, the company open sourced kepler.gl, a data-agnostic, high-performance web-based application for visual exploration of large-scale geolocation datasets. Kepler is currently used by 75,000 Uber-external users and received over 2,500 GitHub stars.

The experience of building the business intelligence platform has been nothing short of extraordinary. In spite of having created multiple tools, the engineering work for compliance with regulations like GDPR has been extremely streamlined because of the way integrations between tools were designed. With a single interface, users are able to write queries across multiple data stores underneath.

Shailesh Chauhan explains how Uber built its business intelligence platform, detailing why the company took a platform approach rather than adding features in a piecemeal fashion. Along the way, Shailesh explores the ways Uber has evolved its goals for the BI Suite, reevaluated strategy, and found creative new ways to meet the needs of its customers—and how lessons learned from the platform are influencing the future direction of BI Suite, including moving toward generating insights automatically using ML.