Improved Disease Classification in Chest X-Rays with Transferred Features from Report Generation

Research output: Chapter in Book/Report/Conference proceedingConference contribution

17 Scopus citations

Abstract

Radiology includes using medical images for detection and diagnosis of diseases as well as guiding further interventions. Chest X-rays are commonly used radiological examinations to help spot thoracic abnormalities or diseases, especially lung-related diseases. However, the reporting of chest x-rays requires experienced radiologists who are often in shortage in many regions of the world. In this paper, we first develop an automatic radiology report generation system. Due to the lack of large annotated radiology report datasets and the difficulty of evaluating the generated reports, the clinical value of such systems is often limited. To this end, we train our report generation network on the small IU Chest X-ray dataset then transfer the learned visual features to classification networks trained on the large ChestX-ray14 dataset and use a novel attention guided feature fusion strategy to improve the detection performance of 14 common thoracic diseases. Through learning the correspondences between different types of feature representations, common features learned by both the report generation and the classification model are assigned with higher attention weights and the weighted visual features boost the performance of state-of-the-art baseline thoracic disease classification networks without altering any learned features. Our work not only offers a new way to evaluate the effectiveness of the learned radiology report generation network, but also proves the possibility of transferring different types of visual representations learned on a small dataset for one task to complement features learned on another large dataset for a different task and improve the model performance.

Original languageEnglish (US)
Title of host publicationInformation Processing in Medical Imaging - 26th International Conference, IPMI 2019, Proceedings
EditorsAlbert C.S. Chung, Siqi Bao, James C. Gee, Paul A. Yushkevich
PublisherSpringer Verlag
Pages125-138
Number of pages14
ISBN (Print)9783030203504
DOIs
StatePublished - 2019
Event26th International Conference on Information Processing in Medical Imaging, IPMI 2019 - Hong Kong, China
Duration: Jun 2 2019Jun 7 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11492 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference26th International Conference on Information Processing in Medical Imaging, IPMI 2019
Country/TerritoryChina
CityHong Kong
Period6/2/196/7/19

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Improved Disease Classification in Chest X-Rays with Transferred Features from Report Generation'. Together they form a unique fingerprint.

Cite this