Sep 9–12, 2019

Creating autonomy for social robots

Dylan Glas (Futurewei Technologies), Phoebe Liu (Figure Eight)
4:50pm5:30pm Wednesday, September 11, 2019
Location: LL21 E/F

Who is this presentation for?

data practitioners, engineers, researchers, roboticist




The technologies enabling social robots are becoming more capable and affordable. Yet, creating autonomy for human-robot interaction is still a non-trivial task, requiring the integration of essential AI building blocks including natural language, vision, navigation, and planning/reasoning. In particular, it is often difficult to quantify human social behavior and rules with algorithms in robots. Our research has centered around deploying social robots to interact with real people in public spaces such as train stations and shopping malls. Building on over 20 years of combined experience with robots ranging from robotic wheelchairs to highly realistic humanlike androids, we present some of our work using AI and machine learning to understand social situations and generate robot behavior, and using learning-by-imitation techniques to reproduce social behavior in an autonomous robot.

We first discuss ways we have adapted our robots to the challenges of real-world environments outside the lab, such as using sensor data to understand the social use of space and predict the movement of pedestrians, and we present algorithms and architectures we have developed to enable social robots to approach people, hold conversational interactions, and provide guidance and services to people in shopping malls, elementary schools, and beyond.

Following that, we will present a data-driven approach for generating speech and motion behavior in an autonomous robot, using learning-by-imitation from unscripted human-human interaction data collected from sensors. This technique requires no manual design, human annotation, modeling, or natural-language understanding, and it is able to autonomously generate socially-appropriate locomotion and speech behaviors for a humanoid robot. Finally, we will discuss some of our key insights into how learning from human behavior is critical for developing autonomous robots for the real world.

Prerequisite knowledge

basic knowledge of machine learning and general interest in social robots

What you'll learn

We highlight practical approaches to enabling human-robot interaction in the real world, including dealing with noisy real-world multimodal data, experiences learned from field trials and understanding what it takes to build a humanoid robot.
Photo of Dylan Glas

Dylan Glas

Futurewei Technologies

Dylan Glas is a roboticist and researcher with over a decade of experience in the field of social human-robot interaction. He was a Guest Associate Professor at Osaka University and a Group Leader and Senior Researcher at the Advanced Telecommunications Research Institute (ATR) in Kyoto, Japan, where he developed frameworks and algorithms for multimodal perception, machine learning, and autonomous behavior generation for a variety of humanoid social robots. He has been featured on international media, including CBS, BBC, CNN, National Geographic, and The Guardian, for his work as the lead architect of ERICA, a highly-humanlike conversational android which is currently operating as a TV news anchor in Japan. He is currently Senior Robotics Software Architect at Futurewei Technologies in San Francisco.

Photo of Phoebe Liu

Phoebe Liu

Figure Eight

Phoebe Liu is currently a machine learning scientist at Figure Eight, an AI and machine-learning startup based in San Francisco. Previously, she was a robotics researcher in Japan, working in Hiroshi Ishiguro Laboratory at Advanced Telecommunications Research Institute International (ATR). At the same time, she earned her Ph.D. at Osaka University (2017). She was involved in projects including enabling conversational social robot to imitate human behaviors, android science, and teleoperation system for semi-autonomous robot.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

For conference registration information and customer service

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

Contact list

View a complete list of O'Reilly AI contacts