Published July 27, 2023
| Version v1.0.0
Dataset
Open
Model outputs for the study "Guidance in Radiology Report Summarization: An Empirical Evaluation and Error Analysis"
Authors/Creators
- 1. University of Duisburg-Essen
- 2. University of Marburg
Description
This resources provides pre-processed input data, model checkpoints and model outputs for experiments on the OpenI dataset in below study.
Jan Trienes, Paul Youssef, Jörg Schlötterer, and Christin Seifert. 2023. Guidance in Radiology Report Summarization: An Empirical Evaluation and Error Analysis. In Proceedings of the 16th International Natural Language Generation Conference (INLG), Prague, Czech Republic. Association for Computational Linguistics.
For more information please refer to the accompanying paper and code repository (https://github.com/jantrienes/inlg2023-radsum).
The data is structured as follows:
data/preprocessed/includes the dataset(s) for each modeloutput/includes one folder for each experiment/model run. The first part of each output path indicates the dataset that was used at inference.- For a mapping between model IDs and results in the paper, see below table. All models were also trained with the background section as input. These are available in directories with the
-bg-qualifier.
| Model name in paper | Output directory |
|---|---|
| Results from Table 2 | |
| OracleExt | openi-unguided/oracle |
| BertExt (Liu and Lapata, 2019) | openi-unguided/bertext-default |
| BertAbs (Liu and Lapata, 2019) | openi-unguided/bertabs-default |
| GSum (Dou et al., 2021) | openi-bertext-default-clip-k1/gsum-default |
| GSum w/ LR-Approx | openi-bertext-default-clip-lrapprox/gsum-default |
| GSum w/ BERT-Approx | openi-bertext-default-clip-bertapprox/gsum-default |
| GSum w/ Thresholding | openi-bertext-default-clip-threshold/gsum-default |
| WGSum (Hu et al., 2021) | openi-wgsum/wgsum-default |
| WGSum+CL (Hu et al., 2022) | openi-wgsum-cl/wgsum-cl-default |
| Results from Table 3 | |
| Fixed (k=1) | openi-unguided/bertext-default-clip-k1 |
| LR-Approx | openi-unguided/bertext-default-clip-lrapprox |
| BERT-Approx | openi-unguided/bertext-default-clip-bertapprox |
| Thresholding | openi-unguided/bertext-default-clip-threshold |
| k = |OracleExt| | openi-unguided/bertext-default-clip-oracle |
| Results from Table 4 | |
| Fixed (Dou et al., 2021) | openi-bertext-default-clip-k1/gsum-default |
| Oracle Length | openi-bertext-default-clip-oracle/gsum-default |
| Oracle Length + Content | openi-oracle/gsum-default |
| Results from Table 5 | |
| BertExt w/ k=[1,5] | openi-unguided/bertext-default-clip-k{1,2,3,4,5} |
| GSum w/ k=[1,5] | openi-bertext-default-clip-k{1,2,3,4,5}/gsum-default |
Files
data.zip
Additional details
Related works
- Is part of
- Conference paper: 10.48550/arXiv.2307.12803 (DOI)
References
- Demner-Fushman et al. (2015) published the OpenI dataset. https://doi.org/10.1093/jamia/ocv080