Presented By O’Reilly and Intel AI
Put AI to work
8-9 Oct 2018: Training
9-11 Oct 2018: Tutorials & Conference
London, UK

Accelerating innovation through analogy mining

Dafna Shahaf (The Hebrew University of Jerusalem)
11:55–12:35 Thursday, 11 October 2018
Models and Methods
Location: King's Suite - Sandringham
Secondary topics:  Deep Learning models, Retail and e-commerce, Text, Language, and Speech
Average rating: ****.
(4.00, 1 rating)

Prerequisite knowledge

  • A basic understanding of word embeddings, text mining, and neural networks

What you'll learn

  • Explore an algorithm that automatically discovers analogies in unstructured data


The availability of large idea repositories (e.g., patents) could significantly accelerate innovation and discovery by providing people inspiration from solutions to analogous problems. However, finding useful analogies in these large, messy real-world repositories remains a persistent challenge for both humans and computers. Previous approaches include costly manually created databases that do not scale or machine learning similarity metrics that struggle to account for structural similarity, which is central to analogy.

Dafna Shahaf explores the viability and value of learning simple structural representations and presents an algorithm that automatically discovers analogies in unstructured data. The approach combines crowdsourcing and recurrent neural networks to extract purpose and mechanism vector representations from product descriptions. Dafna demonstrates that these learned vectors allow us to find analogies with higher precision and recall than traditional methods. In an ideation experiment, analogies retrieved by these models significantly increased people’s likelihood of generating creative ideas.

Photo of Dafna Shahaf

Dafna Shahaf

The Hebrew University of Jerusalem

Dafna Shahaf is an assistant professor in the School of Computer Science and Engineering at the Hebrew University of Jerusalem. Dafna’s research deals with making sense of massive amounts of data. She designs algorithms that help people understand the underlying structure of complex topics, connect the dots between pieces of information, and turn data into insight. She is especially interested in unlocking the potential of the many digital traces left by human activity to understand and emulate human characteristics (e.g., creativity). Previously, she was a postdoctoral fellow at Microsoft Research and Stanford University. Dafna’s work has received multiple awards, including Best Research Paper at KDD’17 and KDD’10 and the IJCAI Early Career Award. She holds a PhD from Carnegie Mellon University.