Presented By O’Reilly and Intel Nervana
Put AI to work
September 17-18, 2017: Training
September 18-20, 2017: Tutorials & Conference
San Francisco, CA

Using deep learning and Google Street View to estimate the demographic makeup of the US

Timnit Gebru (Microsoft Research)
4:50pm–5:30pm Tuesday, September 19, 2017
Location: Grand Ballroom
Secondary topics:  Transportation and autonomous vehicles
Average rating: ****.
(4.75, 4 ratings)

What you'll learn

  • Explore a solution that leverages Google Street View images and a computer vision pipeline to predict income, carbon emission, crime rates, and other city attributes from a single source of publicly available data

Description

Targeted socioeconomic policies require an accurate understanding of a country’s demographics. However, this traditional method of collecting surveys across many years is costly and labor intensive. For example, the US currently spends more than $1 billion a year gathering census data such as race, gender, education, occupation and unemployment rates. Data- and machine learning-driven approaches are cheaper and faster—with the potential ability to detect trends in close to real time. Timnit Gebru shares a solution that leverages Google Street View images and a computer vision pipeline to predict income, carbon emission, crime rates, and other city attributes from a single source of publicly available data.

Timnit and her team first detected cars in 50 million images across 200 of the largest US cities and trained a model to determine demographic attributes using the detected cars. To facilitate this work, they used a graph-based algorithm to collect the largest and most challenging fine-grained dataset to date, consisting of over 2,600 classes of cars comprised of images from Google Street View and other web sources. The prediction results correlate well with ground truth income (r=0.82), race, education, voting, sources investigating crime rates, income segregation, per capita carbon emission, and other market research. The solution also reveals interesting relationships between cars and neighborhoods, enabling the first large-scale sociological analysis of cities using computer vision techniques.

Photo of Timnit Gebru

Timnit Gebru

Microsoft Research

Timnit Gebru is a postdoctoral researcher at Microsoft New York working in the fairness accountability, transparency, and ethics (FATE) group, where she is working on how to take dataset bias into account while designing machine learning algorithms and the ethical considerations underlying any data mining project. Previously, Timnit worked at Apple, where she designed circuits and signal-processing algorithms for various Apple products, including the first iPad. She also spent an obligatory year as an entrepreneur (as all Stanford undergrads seem to do). Timnit is a PhD candidate in the Stanford Artificial Intelligence Laboratory, where she is studying computer vision under Fei-Fei Li. Her research focuses on data mining large-scale, publicly available images to gain sociological insight and the computer vision problems that arise as a result and has been covered by the Economist and other publications.