Sep 9–12, 2019

Transfer learning NLP: Machine reading comprehension for question answering

Anusua Trivedi (Microsoft)
2:35pm3:15pm Thursday, September 12, 2019
Location: 230 C
Average rating: **...
(2.33, 3 ratings)

Who is this presentation for?

  • Software developers and data scientists




Modern machine learning models, especially deep neural networks, often significantly benefit from transfer learning. In computer vision, deep convolutional neural networks trained on a large image classification dataset such as ImageNet have proven useful for initializing models on other vision tasks, such as object detection. But we still have to learn how we can leverage the transfer leaning technique for text.

QA is a long-standing challenge in natural language processing (NLP), and the community has introduced several paradigms and datasets for the task over the past few years. These paradigms differ in the type of questions and answers and the size of the training data from a few hundreds to millions of examples. For human beings, reading comprehension is a basic daily task. As early as elementary school, many of us can read an article and answer questions about its key ideas and details. But for AI, full reading comprehension is still an elusive goal. Therefore, building machines that can perform MRC is of great interest. Recently, several researchers have explored various approaches to attack the MRC transfer learning problem. Their work has been a key step toward developing some scalable QA solutions to extend MRC to a wider range of domains.

Anusua Trivedi details a comprehensive study of existing text transfer learning literature in the research community. She explores popular MRC algorithms and evaluates and compares the performance of a transfer learning approach for creating a QA system for a book corpus using pretrained MRC models. For evaluation, the performance of the document-QA model outperforms that of other transfer learning approaches like bidirectional attention flow (BIDAF), ReasoNet, and R-NET models. She compared the performance of fine-tuning learning approaches for creating a QA corpus for this book using a couple of these pretrained MRC models. The performance of the OpenNMT model outperformed that of the SynNet model.

What you'll learn

  • Learn about automated QA, transfer learning, and NLP
Photo of Anusua Trivedi

Anusua Trivedi


Anusua Trivedi is a senior data scientist lead at Microsoft. She works on AI for Good—developing advanced deep learning models and AI solutions for humanitarian causes. Her focus is AI for healthcare, where she explores how AI can help make healthcare more affordable and accessible to everyone around the world. Previously, Anusua has held positions with UT Austin and the University of Utah. Anusua is a frequent speaker at machine learning and AI conferences.

Comments on this page are now closed.


Diego Klabjan | Professor
09/12/2019 8:01am PDT

Horrible talk. This is an intro to deep learning and not transfer NLP. A complete waste of time.

  • Intel AI
  • O'Reilly
  • Amazon Web Services
  • IBM Watson
  • Dataiku
  • Dell Technologies
  • Intuit
  • Gamalon
  • Hewlett Packard Enterprise
  • MapR Technologies
  • Sisu Data
  • Intuit

Contact us

For conference registration information and customer service

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

For media/analyst press inquires