September 26-27, 2016
New York, NY

How to make robots empathetic to human feelings in real time

Pascale Fung (The Hong Kong University of Science and Technology)
11:00am–11:40am Monday, 09/26/2016
Implementing AI
Location: 3D09 Level: Advanced
Average rating: ****.
(4.33, 3 ratings)

Prerequisite knowledge

  • Familiarity with machine learning and signal processing
  • What you'll learn

  • Understand an approach to recognizing emotion and sentiment from speech in real time
  • Description

    Emotion recognition from speech previously consisted of feature engineering and machine learning, where the first stage caused a delay in decoding time. Pascale Fung describes an approach to enable an interactive dialogue system to recognize user emotion and sentiment in real time. This approach achieves an impressive average of 65.7% accuracy on six emotion categories, a 4.5% improvement when compared to the conventional feature-based SVM classification. Pascale also outlines a separate CNN-based sentiment analysis module that recognizes sentiments from speech recognition results, with a 74.8 F-measure on human-machine dialogues when trained with out-of-domain data. These modules allow otherwise conventional dialogue systems to have “empathy” and answer users while being aware of their emotion and intent.

    Photo of Pascale Fung

    Pascale Fung

    The Hong Kong University of Science and Technology

    Pascale Fung is a professor in the Department of Electronic & Computer Engineering at the Hong Kong University of Science & Technology. She is an elected fellow of the Institute of Electrical and Electronic Engineers (IEEE) for her contributions to human-machine interactions and an elected fellow of the International Speech Communication Association for fundamental contributions to the interdisciplinary area of spoken language human-machine interactions. She is keenly interested in promoting AI research for the betterment of the humanity, including AI for ethical fintech and medical practices. Pascale has recently become a partner in the Partnership on AI, an organization of top AI players in the industry and academia focusing on promoting AI to benefit people and the society. She is a member of the Global Future Council on Artificial Intelligence and Robotics, a think tank of the World Economic Forum, and blogs for the forum’s online publication agenda. Pascale has been recognized as one of 2017’s Outstanding Women Professionals and a Woman of Hope in 2014. She holds a PhD in computer science from Columbia University. She is a fluent speaker of seven European and Asian languages.