Credibility Assessment of Textual Claims on the Web

By Kashyap Popat, Subhabrata Mukherjee, Jannik Strotgen, Gerhard Weikum. 2016

201 citations recorded by the source corpus.

There is an increasing amount of false claims in news, social media, and other web sources. While prior work on truth discovery has focused on the case of checking factual statements, this paper addresses the novel task of assessing the credibility of arbitrary claims made in natural-language text - in an open-domain setting without any assumptions about the structure of the claim, or the community where it is made. Our solution is based on automatically finding sources in news and social media, and feeding these into a distantly supervised classifier for assessing the credibility of a claim (i.e., true or fake). For inference, our method leverages the joint interaction between the language of articles about the claim and the reliability of the underlying web sources. Experiments with claims from the popular website snopes.com and from reported cases of Wikipedia hoaxes demonstrate the viability of our methods and their superior accuracy over various baselines.

Subfields: Artificial Intelligence, Information Systems, Sociology and Political Science, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Sociology and Political Science

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Survey of Hallucination in Natural Language Generation (2022, background): “, web traffic, PageRank or URL structure) have been proposed to deal with this trustworthiness issue [7, 138, 139].”
Predicting Factuality of Reporting and Bias of News Media Sources (2018, methodology): “…news media has often been estimated automatically based on the general stance of the target medium with respect to known manually fact-checked claims, without access to gold labels about the overall medium-level factuality of reporting (Mukherjee and Weikum, 2015; Popat et al., 2016, 2017, 2018).”
FANG: Leveraging Social Context for Fake News Detection Using Graph Representation (2020, background): “They typically learn a Euclidean transformation of the social entity features that best approximates the fake news prediction [32].”
MVAE: Multimodal Variational Autoencoder for Fake News Detection (2019, background): “Popat et al. [19] use stance and language stylistic features like as-sertive verbs, discourse markers, etc. to assess the credibility of a claim.”
Discovering Biased News Articles Leveraging Multiple Human Annotations (2020, background): “Given the vast amount of news we consume on a day-to-day basis, ensuring information quality and credibility (Popat et al., 2016) becomes increasingly crucial, because we need accessto accurateandreliable newsstories.”
FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured information (2021, background): “…either leave out evidence and justiﬁcations entirely [Wang, 2017], rely on search engines which risk including irrelevant or misleading evidence [Popat et al., 2016, Baly et al., 2018, Augenstein et al., 2019], or bypass the retrieval challenge entirely by extracting evidence directly from the…”
Automated Fact-Checking for Assisting Human Fact-Checkers (2021, background): “Non-explainable approaches make a prediction based on the content of documents retrieved from the Web [Popat et al., 2016; Karadzhov et al., 2017; Augenstein et al., 2019], or on social media by modeling the message and its propagation, the users and their reactions over time, links to media sites, ”
Automatic Detection of Fake News (2017, unknown): “, 2017) and fact-checking websites such as ”politiFact” (Wang, 2017) and “Snopes” (Popat et al., 2016).”
Explainable Fact Checking with Probabilistic Answer Set Programming (2019, methodology): “For claims for which no rule can be executed, we resort to a Web text mining system [17].”
SemEval-2019 Task 8: Fact Checking in Community Question Answering Forums (2019, methodology): “The contrastive submissions, the used for similar problems, e.g., Ma et al. (2015) experimented with 226 rumors for rumor detection, and Popat et al. (2016) used 100 Wiki hoaxes for credibility assessment of textual claims.”
DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning (2018, methodology): “On the other hand, Popat et al. (2017) considers external evidence in the form of other articles (retrieved from the Web) that confirm or refute a claim, and jointly assesses the language style (using subjectivity lexicons), the trustworthiness of the sources, and the credibility of the claim.”