Design the Future
January 19–20, 2016: Training
January 20–22, 2016: Conference
San Francisco, CA

How we talk and how machines listen

Abi Jones (Google)
3:45pm Thursday, 01/21/2016
The new fundamentals
Location: Generals Residence Dining
Average rating: ****.
(4.73, 11 ratings)

The English word “run” has over 100 meanings. It can be a noun, verb, or adjective. You can run with a bad crowd, run a river, run to the store, or ride a train that runs from San Francisco to Los Angeles, but each of those meanings is easy to understand and interpret based on your years of experience listening to and speaking with other people. You are an expert at combining semantic information, syntax, and context into an understanding of speech, and you can do so in milliseconds.

Computers can analyze speech even more quickly than you can but do not have the deep contextual understanding that comes with years of being human. Instead, human-computer conversation takes place in an alternate reality where people and machines make entirely different sets of assumptions about their respective capabilities and intentions. Abi Jones compares human-to-human and human-computer conversation and interaction to explain how their differences impact system design.

Abi dives into semantic understanding, syntax, and context to demonstrate how machines interpret and act on speech, explore why human speech is vague and frustrating for machines, hear and see how spoken dialogue really sounds, and explain how spontaneous speech has wrought havoc on flight-scheduling systems. Abi covers the spectrum of natural language interaction and examines how research teams create the language sets that voice systems learn from, including overcoming elicitation bias and the challenges of crowdsourcing natural language corpora. By the end of the session, you’ll know what makes for a great human-computer speech interaction and why it’s more enjoyable and addictive to talk to a 1960s chatbot than any intelligent assistant available today.

Photo of Abi Jones

Abi Jones


Abi Jones is a designer and sprint master at Google Fiber, where she helps teams answer critical business questions through design, prototyping, and testing ideas with customers. Abi specializes in product development via storyboarding. She illustrated the second edition of Understanding Your Users.