Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation

By Fenglin Liu, Xian Wu, Shen Ge, Wei Fan, Yuexian Zou. Computer Vision and Pattern Recognition, 2021

393 citations recorded by the source corpus.

Automatically generating radiology reports can improve current clinical practice in diagnostic radiology. On one hand, it can relieve radiologists from the heavy burden of report writing; On the other hand, it can remind radiologists of abnormalities and avoid the misdiagnosis and missed diagnosis. Yet, this task remains a challenging job for data-driven neural networks, due to the serious visual and textual data biases. To this end, we propose a Posterior-and-Prior Knowledge Exploring-and-Distilling approach (PPKED) to imitate the working patterns of radiologists, who will first examine the abnormal regions and assign the disease topic tags to the abnormal regions, and then rely on the years of prior medical knowledge and prior working experience accumulations to write reports. Thus, the PPKED includes three modules: Posterior Knowledge Explorer (PoKE), Prior Knowledge Explorer (PrKE) and Multi-domain Knowledge Distiller (MKD). In detail, PoKE explores the posterior knowledge, which provides explicit abnormal visual regions to alleviate visual data bias; PrKE explores the prior knowledge from the prior medical knowledge graph (medical knowledge) and prior radiology reports (working experience) to alleviate textual data bias. The explored knowledge is distilled by the MKD to generate the final reports. Evaluated on MIMIC-CXR and IU-Xray datasets, our method is able to outperform previous state-of-the-art models on these two datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/279925dbe2ebe4e38d6cfa646f726aa8d6d9a122

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Giuseppe Fiameni
  2. Marcella Cornia
  3. Lorenzo Baraldi
  4. Rita Cucchiara
  5. Silvia Cascianelli
  6. Matteo Stefanini
  7. Flavius Frasincar
  8. Gianni Brauwers

In Computer Vision and Pattern Recognition

  1. Rita Cucchiara
  2. Marcella Cornia
  3. Lorenzo Baraldi
  4. Giuseppe Fiameni
  5. Matteo Stefanini
  6. Silvia Cascianelli
  7. Kristen Grauman
  8. Yi Yang

Citations in the corpus