Evidence-based Factual Error Correction
By James Thorne, Andreas Vlachos. 2020
70 citations recorded by the source corpus.
This paper introduces the task of factual error correction: performing edits to a claim so that the generated rewrite is better supported by evidence. This extends the well-studied task of fact verification by providing a mechanism to correct written texts that are refuted or only partially supported by evidence. We demonstrate that it is feasible to train factual error correction systems from existing fact checking datasets which only contain labeled claims accompanied by evidence, but not the correction. We achieve this by employing a two-stage distant supervision approach that incorporates evidence into masked claims when generating corrections. Our approach, based on the T5 transformer and using retrieved evidence, achieved better results than existing work which used a pointer copy network and gold evidence, producing accurate factual error corrections for 5x more instances in human evaluation and a .125 increase in SARI score. The evaluation is conducted on a dataset of 65,000 instances based on a recent fact verification shared task and we release it to enable further work on the task.
Subfields: Artificial Intelligence, Information Systems, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Luyu Gao
- Sebastian Riedel
- Andreas Vlachos
- Kelvin Guu
- Luke S. Zettlemoyer
- Wen-tau Yih
- Percy Liang
- Hannaneh Hajishirzi
In Information Systems
- Luyu Gao
- Sebastian Riedel
- Amrith Krishna
- Panupong Pasupat
- Ni Lao
- Andreas Vlachos
- Kelvin Guu
- Zhuyun Dai
In Natural Language Processing
- Luyu Gao
- Sebastian Riedel
- Andreas Vlachos
- Panupong Pasupat
- Kelvin Guu
- Hongrae Lee
- Ni Lao
- Amrith Krishna
Citations in the corpus
- ProoFVer: Natural Logic Theorem Proving for Fact Verification (2021, methodology): “, 2020), correct factual errors (Thorne and Vlachos, 2021b; Schuster et al., 2021) or perform rule discovery (Ahmadi et al.”
- RARR: Researching and Revising What Language Models Say, Using Language Models (2022, methodology): “This is likely due to EFEC’s training data, which was limited to single sentences.”
- Attributed Text Generation via Post-hoc Research and Revision (2022, methodology): “This is likely due to EFEC’s training data, which was limited to single sentences.”
- Factual Error Correction for Abstractive Summaries Using Entity Retrieval (2022, unknown): “For this reason, previous works (Cao et al., 2020; Zhu et al., 2021; Thorne and Vlachos, 2021) have introduced post-editing systems to alleviate these factual errors in the summary.”