Diagnose and explain: Neural X-ray diagnosis with visual and textual evidence
Who is this presentation for?
- NLP researchers playing with clinical data, radiologists, and healthcare specialists
In the past few months, Mila conducted a study to understand how deep learning models are being used in clinical settings, especially in radiology. The general opinion is that because they sometimes outperform humans at predicting diseases, deep learning models occupy an important place within radiologists’ workflows. This is unfortunately not true when it comes to X-ray interpretation.
The study found that radiologists don’t rely on diagnostics produced by black box systems, when they don’t have access to clinical findings supporting these diagnostics. This suggests that predicting diseases with no form of clinical explanation is of little interest to radiologists, regardless of prediction accuracy. In the meantime, a radiology report contains different sections, including “clinical findings” and “impression,” which constitute textual evidence supporting a diagnosis. Since there exists large free-text databases with such data, the researchers studied whether a language model can be trained to generate these sections at test time, with satisfying clinical accuracy.
Wisdom d’Almeida dives into what makes this task challenging from a natural language modeling point of view and presents a novel approach to optimize language models for clinical pertinence. He details the design and training of a medical report generation model with TensorFlow and its testing with a TensorFlow.js web interface. The dataset used is MIMIC-CXR, a large publicly available database of chest radiographs with free-text radiology reports.
- Experience with language modeling, image captioning, and image classification
What you'll learn
- Learn how to use TensorFlow and Cloud TPUs to train a model for radiology report generation and how to optimize language models for clinical pertinence (making them more "clinically aware")
Wisdom d’Almeida is a Visiting Researcher at Mila, working with Yoshua Bengio and Alain Tapp on fundamental machine learning questions related to consciousness. His other research interests include grounded language learning and AI explainability. In the past, Wisdom worked on natural language understanding for common-sense reasoning, with application to areas such as healthcare—his master’s dissertation was about medical report generation with natural language explanations. Wisdom’s works in AI won a Government of India National Award in 2018. Previously, he interned at Google in San Francisco and demoed at Google Cloud Next 2018. Wisdom holds a master’s degree from KIIT in India and a BS from Université de Lomé in Togo, where he grew up. In his spare time, you can see him struggling with his vocal cords and his guitar strings.
Leave a Comment or Question
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
For conference registration information and customer service
For more information on community discounts and trade opportunities with O’Reilly conferences
For information on exhibiting or sponsoring a conference
For media/analyst press inquires