A Survey on Automated Fact-Checking

By Zhijiang Guo, Michael Sejr Schlichtkrull, Andreas Vlachos. 2021

757 citations recorded by the source corpus.

Fact-checking has become increasingly important due to the speed with which both information and misinformation can spread in the modern media ecosystem. Therefore, researchers have been exploring how fact-checking can be automated, using techniques based on natural language processing, machine learning, knowledge representation, and databases to automatically predict the veracity of claims. In this paper, we survey automated fact-checking stemming from natural language processing, and discuss its connections to related tasks and disciplines. In this process, we present an overview of existing datasets and models, aiming to unify the various definitions given and identify common concepts. Finally, we highlight challenges for future research.

Subfields: Artificial Intelligence, Sociology and Political Science, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “directions include automated fact-checking, propaganda, or fake news detection for which several annotated datasets (Oshikawa et al., 2020; Martino et al., 2020; Zhou and Zafarani, 2020; Guo et al., 2022; Huang et al., 2022) and shared tasks (Thorne et al.”
RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models (2023, background): “Please refer to Appendix C for more statistical in-316 formation about these annotations.”
DialFact: A Benchmark for Fact-Checking in Dialogue (2021, background): “Fact-checking has been explored in mediums such as Wikipedia passages, tables, scientific articles, news articles, and social media claims (Guo et al., 2021).”
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing (2023, background): “[54] Zhijiang Guo, Michael Schlichtkrull, and Andreas Vlachos.”
Knowledge Conflicts for LLMs: A Survey (2024, neutral): “Conflicting information can stem from misinformation, outdated facts, or partially correct data (Uscinski and Butler, 2013; Guo et al., 2022).”
Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks (2023, background): “…(i) state-of-the-art LLMs have dramatically improved in their instruction-following ability (Ouyang et al., 2022; Wei et al., 2021; Sanh et al., 2021; Iyer et al., 2022); and (ii) Zellers et al., 2019; Markov et al., 2022; Pavlopoulos et al., 2020; Gehman et al., 2020; Guo et al., 2022).”
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2023, positive): “This methodology closely aligns with the workflow of fact-checking tasks, as delineated by (Guo et al., 2022).”
ExpertQA: Expert-Curated Questions and Attributed Answers (2023, background): “…2023; Azaria and Mitchell, 2023; Min et al., 2023), which operate mostly in a black-box fashion, and resource-enriched approaches (Thorne et al., 2018; Guo et al., 2022; Chen et al., 2023; Feng et al., 2023), that verify statements by checking their validity against external databases or resources.”
Character-LLM: A Trainable Agent for Role-Playing (2023, background): “Further, as trained with wide worldwide knowledge, it is very likely that LLM-based agents will produce hallucinations that violate their characters (Kryscinski et al., 2020; Guo et al., 2022; Ji et al., 2023).”
When retrieval outperforms generation: Dense evidence retrieval for scalable fake news detection (2025, unknown): “The exponential growth of misinformation across digital platforms presents an urgent challenge to information integrity and societal discourse (Guo et al., 2022).”
Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generation (2025, unknown): “In this section, we introduce semantic isotropy , a measure of semantic variation in long-form text generations, and use it to motivate semantic isotropy scoring , a method that expresses the level of semantic isotropy in long-form responses and can serve as a proxy for nonfactuality.”