For a company looking to provide delightful user experiences, it is of paramount importance to take care of any customer issues. Uber has implemented COTA, a system to improve the speed and reliability of customer support for end users through automated ticket classification and answer selection for support representatives. By improving speed and reliability, COTA also helps reduce customer support operations costs.
Piero Molino, Huaixiu Zheng, and Yi-Chia Wang describe how Uber built COTA with traditional and deep learning models and share the lessons learned along the way. Piero, Huaixiu, and Yi-Chia cover two machine learning (ML) and natural language processing (NLP) systems: one relying on feature engineering (COTA v1) and the other exploiting raw signals through deep learning architectures (COTA v2). COTA v1 converts a multiclassification task into a ranking problem, demonstrating significantly better performance in the case of thousands of classes. COTA v2 uses an encoder-combiner-decoder, a novel deep learning architecture that allows for heterogeneous input and output feature types and injection of prior knowledge through network architecture choices. They offer a comparison of these models and their variants on the task of ticket classification and answer selection and analyze their inner workings and shortcomings. Along the way, they explain how Uber is productionizing its models and describe the A/B test the company ran to evaluate the real-world impact of COTA, which showed a 10% reduction in issue resolution time without reducing customer satisfaction.
Piero Molino is a senior research scientist at Uber AI, where he focuses on machine learning for language and dialogue. Previously, he founded QuestionCube, a startup that built a framework for semantic search and QA, and worked on learning to rank at Yahoo Labs in Barcelona, on natural language processing with deep learning at IBM Watson in New York, and on grounded language understanding at Geometric Intelligence (acquired by Uber). Piero holds a PhD on question answering from the University of Bari, Italy.
Huaixiu Zheng is a data scientist at Uber, where he’s a major contributor to several ongoing efforts using deep learning-based NLP, ML, and AI technologies to empower intelligent business operations. Huaixiu has made significant contributions in the fields of quantum waveguide QED, quantum phase transition in dissipative environments, and photonic quantum computation. Previously, he was a postdoctoral researcher at Yale University, where he worked on quantum error corrections and topological quantum computation. He’s published more than 25 journal and conference papers in prestigious journals such as Nature, Nature Physics, and Physical Review Letters and has more than 1,000 citations. He received prestigious academic and industrial awards, including the Chinese Government Award for Outstanding Self-Financed Students Abroad, the John T. Chambers Scholarship, a second-place award from the SPIE-AAPM-NCI Prostate MR Classification Challenge, a second-place award for the SPIE-AAPM-NCI Prostate MR Gleason Grade Group Challenge, and the second prize (as part of team Future Lifecare) of the 8th Intelligent System Summit & TEEC Cup Startup Contest. He holds a PhD in quantum physics and quantum computation from Duke University.
Yi-Chia Wang is a data scientist on the applied machine learning team at Uber. Yi-Chia’s research interests and skills combine language processing technologies, machine learning methodologies, and social science theories to analyze large-scale data and understand social behavior in online environments. Previously, she worked on question answering and information retrieval. Yi-Chia holds a PhD from the Language Technologies Institute, part of the School of Computer Science at Carnegie Mellon University. Her thesis developed a new machine learning model to measure self-disclosure in social networking site communication at scale and used it to better understand the contexts in which users disclose more or less about themselves.
©2018, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com