Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-ray Reports

By Baoyu Jing, Zeya Wang, Eric P. Xing. 2019

182 citations recorded by the source corpus.

Chest X-Ray (CXR) images are commonly used for clinical screening and diagnosis. Automatically writing reports for these images can considerably lighten the workload of radiologists for summarizing descriptive findings and conclusive impressions. The complex structures between and within sections of the reports pose a great challenge to the automatic report generation. Specifically, the section Impression is a diagnostic summarization over the section Findings; and the appearance of normality dominates each section over that of abnormality. Existing studies rarely explore and consider this fundamental structure information. In this work, we propose a novel framework which exploits the structure information between and within report sections for generating CXR imaging reports. First, we propose a two-stage strategy that explicitly models the relationship between Findings and Impression. Second, we design a novel co-operative multi-agent system that implicitly captures the imbalanced distribution between abnormality and normality. Experiments on two CXR report datasets show that our method achieves state-of-the-art performance in terms of various evaluation metrics. Our results expose that the proposed approach is able to generate high-quality medical reports through integrating the structure information.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Generation, Medical Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohamed Elhoseiny
  2. Yuhao Zhang
  3. Boyang Albert Li
  4. Curtis P. Langlotz
  5. Dan Jurafsky
  6. Jun Chen
  7. Kai Yi
  8. Xian Wu

In Computer Vision and Pattern Recognition

  1. Dan Jurafsky
  2. Mohamed Elhoseiny
  3. Boyang Albert Li
  4. Kai Yi
  5. Xian Wu
  6. Yan Song
  7. Fenglin Liu
  8. Han Guo

In Natural Language Generation

  1. Dan Jurafsky
  2. Yan Song
  3. Xiang Wan
  4. Zhihong Chen

In Medical Imaging

  1. Kai Yi
  2. Boyang Albert Li
  3. Dan Jurafsky
  4. Yan Song
  5. Han Guo
  6. Xiang Wan
  7. Mohamed Elhoseiny
  8. Yaling Shen

Citations in the corpus