Explainable Automated Fact-Checking for Public Health Claims

By Neema Kotonya, Francesca Toni. 2020

357 citations recorded by the source corpus.

Fact-checking is the task of verifying the veracity of claims by assessing their assertions against credible evidence. The vast majority of fact-checking studies focus exclusively on political claims. Very little research explores fact-checking for other topics, specifically subject matters for which expertise is required. We present the first study of explainable fact-checking for claims which require specific expertise. For our case study we choose the setting of public health. To support this case study we construct a new dataset PUBHEALTH of 11.8K claims accompanied by journalist crafted, gold standard explanations (i.e., judgments) to support the fact-check labels for claims. We explore two tasks: veracity prediction and explanation generation. We also define and evaluate, with humans and computationally, three coherence properties of explanation quality. Our results indicate that, by training on in-domain data, gains can be made in explainable, automated fact-checking for claims which require specific expertise.

Subfields: Artificial Intelligence, General Social Sciences, Sociology and Political Science, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Social Sciences

In Sociology and Political Science

In AI Ethics, Fairness & Society

Citations in the corpus

ProoFVer: Natural Logic Theorem Proving for Fact Verification (2021, background): “, 2020), generate summaries (Kotonya and Toni, 2020; Atanasova et al., 2020), correct factual errors (Thorne and Vlachos, 2021b; Schuster et al.”
LongForm: Effective Instruction Tuning with Reverse Instructions (2023, neutral): “…(Hipson and Mohammad, 2020), table to text generation (Iyyer et al., 2017), story generation (Sap et al., 2020; Orbach and Goldberg, 2020; Lin et al., 2019), summarization (Fabbri et al., 2019; Kornilova and Eidelman, 2019; Bražin-skas et al., 2020), and fact generation (Kotonya and Toni, 2020).”
e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks (2021, background): “First adoptions of NLEs have been in image classification [23] and were further extended to self-driving cars [26], visual question-answering [37], and natural language processing [11; 39; 9; 6; 27; 35; 28; 12].”
Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing (2021, background): “11 [100] commonsense QA crowd 10,962 ECQA [2] commonsense QA crowd 10,962 SEN-MAKING [124] commonsense validation students + authors 2,021 CHANGEMYVIEW [10] argument persuasiveness crowd 37,718 WINOWHY [144] pronoun coreference resolution crowd 273 (5) SBIC [111] social bias inference crowd 48,923 (”
Teach Me to Explain: A Review of Datasets for Explainable NLP (2021, background): “, 2020) social bias inference crowd 48,923 (1-3) PUBHEALTH (Kotonya and Toni, 2020b) verifying claims from text auto 11,832 Wang et al.”
Evidence-based Factual Error Correction (2020, background): “With potentially sensitive applications, recent works have focused on building explain-able variants of fact checking (Atanasova et al., 2020; Stammbach and Ash, 2020; Kotonya and Toni, 2020).”
Explainable Automated Fact-Checking: A Survey (2020, background): “Shu et al. (2019), Gad-Elrab et al. (2019), Atanasova et al. (2020a), Kotonya and Toni (2020), and Yang et al. (2019) employ human annotators to assess the quality of explanations generated by their models.”
When retrieval outperforms generation: Dense evidence retrieval for scalable fake news detection (2025, unknown): “RAWFC LIAR-RAW P R F1 P R F1 Traditional approach dEFEND (Shu et al., 2019) 44.90 43.20 44.00 23.00 18.50 20.50 SentHAN (Ma et al., 2019) 45.70 45.50 45.60 22.60 20.00 21.20 SBERT-FC (Kotonya and Toni, 2020a,b) 51.10 46.00 48.40 24.10 22.10 23.10 CofCED (Yang et al., 2022) 53.00 51.00 52.00 29.50 29”