Diagnose and explain: Neural X-ray diagnosis with visual and textual evidence

Description

In the past few months, Mila conducted a study to understand how deep learning models are being used in clinical settings, especially in radiology. The general opinion is that because they sometimes outperform humans at predicting diseases, deep learning models occupy an important place within radiologists’ workflows. This is unfortunately not true when it comes to X-ray interpretation.

The study found that radiologists don’t rely on diagnostics produced by black box systems, when they don’t have access to clinical findings supporting these diagnostics. This suggests that predicting diseases with no form of clinical explanation is of little interest to radiologists, regardless of prediction accuracy. In the meantime, a radiology report contains different sections, including “clinical findings” and “impression,” which constitute textual evidence supporting a diagnosis. Since there exists large free-text databases with such data, the researchers studied whether a language model can be trained to generate these sections at test time, with satisfying clinical accuracy.

Wisdom d’Almeida dives into what makes this task challenging from a natural language modeling point of view and presents a novel approach to optimize language models for clinical pertinence. He details the design and training of a medical report generation model with TensorFlow and its testing with a TensorFlow.js web interface. The dataset used is MIMIC-CXR, a large publicly available database of chest radiographs with free-text radiology reports.