Presented By O'Reilly and Cloudera
Make Data Work
March 28–29, 2016: Training
March 29–31, 2016: Conference
San Jose, CA

Secrets of natural language UIs: Translating English into computer actions

Joseph Turian (Workday), Alex Nisnevich (Bayes Impact)
4:20pm–5:00pm Thursday, 03/31/2016
Data Innovations

Location: LL21 E/F
Tags: featured
Average rating: ****.
(4.78, 9 ratings)

Next-gen UIs will allow people to use plain English to interact with software. However, current published research focuses on abstract understanding, not on translating English into concrete software actions. Rich interactions with software require a complicated interface. For example, imagine trying to query an enterprise database to “find open opportunities owned by my team.” Many people don’t know SQL, and desktop drag-and-drop UIs are simply a thin veneer over SQL. And on a mobile device? Forget it.

English is a compelling interface for communicating with a computer. It allows nonprogrammers to have deeper interactions more quickly and intuitively than with traditional UIs. The next generation of user interfaces will be driven by semantic understanding of natural language input. However, existing “semantic parsing” frameworks tend to be unwieldy and optimized for academic rather than real-world usage. The solution: build your own. But how?

Joseph Turian and Alex Nisnevich outline UPSHOT’s English-to-SQL semantic parser and demonstrate how to build your own English-to-“your software application” parser. Aimed at an audience that is technical but unfamiliar with natural language parsing (NLP), Joseph and Alex’s talk describes how UPSHOT’s algorithm can be used to translate plain English expressions into SQL queries and explains how this approach can be generalized to other software applications beyond database querying. Along the way, Joseph and Alex will highlight potential pitfalls and challenges with this approach and explain the solutions that UPSHOT used in practice.

You will walk away from this talk with the knowledge you need to write your own semantic parser that can take English input and turn it into something meaningful that software can understand.

Photo of Joseph Turian

Joseph Turian


Joseph Turian is currently a principal engineer at Workday. He headed the machine-learning consultancy MetaOptimize LLC and founded the startup UPSHOT (acquired by Workday), which allowed users to query enterprise data from a mobile device using natural language.

Joseph holds a PhD in computer science from New York University, where his research focused on machine learning (ML) and natural language processing (NLP). He has over 14 refereed academic publications in top NLP and ML conferences, with more than 1,700 citations. During his graduate studies, he developed a fast, large-scale machine-learning method for parsing natural language. He received his AB from Harvard University in 2001. He is an advocate for open-notebook science, releasing his research code on his GitHub, and for broader scientific collaboration through the Internet.

Photo of Alex Nisnevich

Alex Nisnevich

Bayes Impact

Alex Nisnevich is a data scientist at Bayes Impact. Previously, he worked on machine-learning pipelines at Workday and built natural language interfaces for databases at UPSHOT. He received his MS in NLP at UC Berkeley.