Transfer learning NLP: Machine reading comprehension for question answering
Who is this presentation for?
- Software developers and data scientists
Modern machine learning models, especially deep neural networks, often significantly benefit from transfer learning. In computer vision, deep convolutional neural networks trained on a large image classification dataset such as ImageNet have proven useful for initializing models on other vision tasks, such as object detection. But we still have to learn how we can leverage the transfer leaning technique for text.
QA is a long-standing challenge in natural language processing (NLP), and the community has introduced several paradigms and datasets for the task over the past few years. These paradigms differ in the type of questions and answers and the size of the training data from a few hundreds to millions of examples. For human beings, reading comprehension is a basic daily task. As early as elementary school, many of us can read an article and answer questions about its key ideas and details. But for AI, full reading comprehension is still an elusive goal. Therefore, building machines that can perform MRC is of great interest. Recently, several researchers have explored various approaches to attack the MRC transfer learning problem. Their work has been a key step toward developing some scalable QA solutions to extend MRC to a wider range of domains.
Anusua Trivedi details a comprehensive study of existing text transfer learning literature in the research community. She explores popular MRC algorithms and evaluates and compares the performance of a transfer learning approach for creating a QA system for a book corpus using pretrained MRC models. For evaluation, the performance of the document-QA model outperforms that of other transfer learning approaches like bidirectional attention flow (BIDAF), ReasoNet, and R-NET models. She compared the performance of fine-tuning learning approaches for creating a QA corpus for this book using a couple of these pretrained MRC models. The performance of the OpenNMT model outperformed that of the SynNet model.
What you'll learn
- Learn about automated QA, transfer learning, and NLP
Anusua Trivedi is a senior data scientist lead at Microsoft. She works on AI for Good—developing advanced deep learning models and AI solutions for humanitarian causes. Her focus is AI for healthcare, where she explores how AI can help make healthcare more affordable and accessible to everyone around the world. Previously, Anusua has held positions with UT Austin and the University of Utah. Anusua is a frequent speaker at machine learning and AI conferences.
Leave a Comment or Question
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
Diversity and Inclusion Sponsor
Premier Exhibitor Plus
R & D and Innovation Track Sponsor
For conference registration information and customer service
For more information on community discounts and trade opportunities with O’Reilly conferences
For information on exhibiting or sponsoring a conference
View a complete list of O'Reilly AI contacts