For a company looking to provide delightful user experiences, it is of paramount importance to take care of any customer issues. Uber has implemented COTA, a system to improve the speed and reliability of customer support for end users through automated ticket classification and answer selection for support representatives. By improving speed and reliability, COTA also helps reduce customer support operations costs.
Piero Molino, Huaixiu Zheng, and Yi-Chia Wang describe how Uber built COTA with traditional and deep learning models and share the lessons learned along the way. Piero, Huaixiu, and Yi-Chia cover two machine learning (ML) and natural language processing (NLP) systems: one relying on feature engineering (COTA v1) and the other exploiting raw signals through deep learning architectures (COTA v2). COTA v1 converts a multiclassification task into a ranking problem, demonstrating significantly better performance in the case of thousands of classes. COTA v2 uses an encoder-combiner-decoder, a novel deep learning architecture that allows for heterogeneous input and output feature types and injection of prior knowledge through network architecture choices. They offer a comparison of these models and their variants on the task of ticket classification and answer selection and analyze their inner workings and shortcomings. Along the way, they explain how Uber is productionizing its models and describe the A/B test the company ran to evaluate the real-world impact of COTA, which showed a 10% reduction in issue resolution time without reducing customer satisfaction.
Piero Molino is a senior research scientist at Uber AI, where he focuses on machine learning for language and dialogue. Previously, he founded QuestionCube, a startup that built a framework for semantic search and QA, and worked on learning to rank at Yahoo Labs in Barcelona, on natural language processing with deep learning at IBM Watson in New York, and on grounded language understanding at Geometric Intelligence (acquired by Uber). Piero holds a PhD on question answering from the University of Bari, Italy.
Huaixiu Zheng is a tech lead on the conversational AI team at Uber, focusing on applications and applied research of natural language processing, deep learning, and conversational AI systems. He’s led several big initiatives at Uber in using AI technologies to empower business applications in the domains of customer support, smart-reply systems, and task-oriented conversational AI systems. He received his PhD in quantum physics and quantum computation from Duke University. He’s published 30+ papers in prestigious journals such as Nature, Nature Physics, and Physical Review Letters and conferences such as KDD.
Yi-Chia Wang is a data scientist on the applied machine learning team at Uber. Yi-Chia’s research interests and skills combine language processing technologies, machine learning methodologies, and social science theories to analyze large-scale data and understand social behavior in online environments. Previously, she worked on question answering and information retrieval. Yi-Chia holds a PhD from the Language Technologies Institute, part of the School of Computer Science at Carnegie Mellon University. Her thesis developed a new machine learning model to measure self-disclosure in social networking site communication at scale and used it to better understand the contexts in which users disclose more or less about themselves.
©2018, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com