Generating Fact Checking Explanations

By Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, Isabelle Augenstein. 2020

241 citations recorded by the source corpus.

Most existing work on automated fact checking is concerned with predicting the veracity of claims based on metadata, social network spread, language used in claims, and, more recently, evidence supporting or denying claims. A crucial piece of the puzzle that is still missing is to understand how to automate the most elaborate part of the process – generating justifications for verdicts on claims. This paper provides the first study of how these explanations can be generated automatically based on available claim context, and how this task can be modelled jointly with veracity prediction. Our results indicate that optimising both objectives at the same time, rather than training them separately, improves the performance of a fact checking system. The results of a manual evaluation further suggest that the informativeness, coverage and overall quality of the generated explanations are also improved in the multi-task model.

Subfields: Artificial Intelligence, Management Science and Operations Research, Sociology and Political Science, Natural Language Processing, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Sociology and Political Science

In Natural Language Processing

In AI Ethics, Fairness & Society

Citations in the corpus

ProoFVer: Natural Logic Theorem Proving for Fact Verification (2021, background): “, 2020), generate summaries (Kotonya and Toni, 2020; Atanasova et al., 2020), correct factual errors (Thorne and Vlachos, 2021b; Schuster et al.”
Explainable Automated Fact-Checking for Public Health Claims (2020, background): “, 2018), and natural language generation (Atanasova et al., 2020).”
Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence (2021, background): “Triggers (Atanasova et al., 2020): A set of 186 FEVER claims paraphrased adversarially to contain universal adversarial triggers (Wallace et al., 2019).”
e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks (2021, background): “First adoptions of NLEs have been in image classification [23] and were further extended to self-driving cars [26], visual question-answering [37], and natural language processing [11; 39; 9; 6; 27; 35; 28; 12].”
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI (2020, background): “Societal and environmental well-being Encourage sustainable and eco-friendly AI Reproducibility checklists Analayzing individual neurons [10] Assess the impact on individuals Fairness checklists Bias exposure [65] Assess the impact on society and democracy Fairness checklists Explanations designed f”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, methodology): “A joint output-explanation model is trained to produce an prediction and simultaneously generate an explanation for the reason of that prediction [14, 93, 109].”
Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanations (2021, background): “However, NLEs are sometimes produced separately from predictions (Marasovic et al., 2020; Brahman et al., 2021; Atanasova et al., 2020), which raises questions about their faithfulness.”
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2023, unknown): “Nevertheless, traditional fact-checking methodologies (Augenstein et al., 2019; Hanselowski et al., 2019; Atanasova et al., 2020) often incorporate simplified assumptions for practicality, leading to discrepancies when applied to complex real-world scenarios.”
Annotating and Modeling Fine-grained Factuality in Summarization (2021, background): “Another recent line of work has looked at fact veriﬁcation (Thorne et al., 2018; Nie et al., 2019; Atanasova et al., 2020).”
TX-Ray: Quantifying and Explaining Model-Knowledge Transfer in (Un-)Supervised NLP (2019, methodology): “Recent explainability methods (Gehrmann et al., 2019; Belinkov and Glass, 2019; Gilpin et al., 2018; Atanasova et al., 2020) Con-neau and Kiela, 2018; Schwarzenberg et al., 2019), or language understanding (Wang et al., 2019; Giulianelli et al., 2018).”
Evidence-based Factual Error Correction (2020, background): “With potentially sensitive applications, recent works have focused on building explain-able variants of fact checking (Atanasova et al., 2020; Stammbach and Ash, 2020; Kotonya and Toni, 2020).”
Explainable Automated Fact-Checking: A Survey (2020, background): “Shu et al. (2019), Gad-Elrab et al. (2019), Atanasova et al. (2020a), Kotonya and Toni (2020), and Yang et al. (2019) employ human annotators to assess the quality of explanations generated by their models.”
Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs (2020, background): “…(Do et al., 2020; Camburu et al., 2018), (visual) QA (Wu and Mooney, 2019; Rajani et al., 2019), arcade games (Ehsan et al., 2019), fact checking (Atanasova et al., 2020), image classification (Hendricks et al., 2018), motivation prediction (Vondrick et al., 2016), algebraic word problems (Ling…”
A Diagnostic Study of Explainability Techniques for Text Classification (2020, background): “Some studies propose the generation of text serving as an explanation, e.g., (Camburu et al., 2018; Lei et al., 2016; Atanasova et al., 2020a).”
When retrieval outperforms generation: Dense evidence retrieval for scalable fake news detection (2025, unknown): “Atanasova (2024) detects fake news independently or jointly with explanations in the multi-task set-up.”