Transfer learning NLP: Machine reading comprehension for question answering

Description

Modern machine learning models, especially deep neural networks, often significantly benefit from transfer learning. In computer vision, deep convolutional neural networks trained on a large image classification dataset such as ImageNet have proven useful for initializing models on other vision tasks, such as object detection. But we still have to learn how we can leverage the transfer leaning technique for text.

QA is a long-standing challenge in natural language processing (NLP), and the community has introduced several paradigms and datasets for the task over the past few years. These paradigms differ in the type of questions and answers and the size of the training data from a few hundreds to millions of examples. For human beings, reading comprehension is a basic daily task. As early as elementary school, many of us can read an article and answer questions about its key ideas and details. But for AI, full reading comprehension is still an elusive goal. Therefore, building machines that can perform MRC is of great interest. Recently, several researchers have explored various approaches to attack the MRC transfer learning problem. Their work has been a key step toward developing some scalable QA solutions to extend MRC to a wider range of domains.

Anusua Trivedi details a comprehensive study of existing text transfer learning literature in the research community. She explores popular MRC algorithms and evaluates and compares the performance of a transfer learning approach for creating a QA system for a book corpus using pretrained MRC models. For evaluation, the performance of the document-QA model outperforms that of other transfer learning approaches like bidirectional attention flow (BIDAF), ReasoNet, and R-NET models. She compared the performance of fine-tuning learning approaches for creating a QA corpus for this book using a couple of these pretrained MRC models. The performance of the OpenNMT model outperformed that of the SynNet model.