A simple but tough-to-beat baseline for the Fake News Challenge stance detection task

By Benjamin Riedel, Isabelle Augenstein, Georgios P. Spithourakis, Sebastian Riedel. 2017

253 citations recorded by the source corpus.

Identifying public misinformation is a complicated and challenging task. An important part of checking the veracity of a specific claim is to evaluate the stance different news sources take towards the assertion. Automatic stance evaluation, i.e. stance detection, would arguably facilitate the process of fact checking. In this paper, we present our stance detection system which claimed third place in Stage 1 of the Fake News Challenge. Despite our straightforward approach, our system performs at a competitive level with the complex ensembles of the top two winning teams. We therefore propose our system as the 'simple but tough-to-beat baseline' for the Fake News Challenge stance detection task.

Subfields: Artificial Intelligence, Sociology and Political Science, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

FEVER: a Large-scale Dataset for Fact Extraction and VERification (2018, methodology): “For a simple well-performing baseline, we selected Riedel et al. (2017)’s submission from the 2017 Fake News Challenge.”
Generating Fact Checking Briefs (2020, neutral): “It has been treated as a classiﬁcation problem (Wang, 2017), often using stance detection (Riedel et al., 2017).”
Predicting Factuality of Reporting and Bias of News Media Sources (2018, methodology): “Stance detection has been addressed as a task in its own right, where models have been developed based on data from the Fake News Challenge (Riedel et al., 2017; Thorne et al., 2017; Mohtarami et al., 2018; Hanselowski et al., 2018), or from SemEval-2017 Task 8 (Derczynski et al., 2017; Dungs et…”
Automated Fact Checking: Task Formulations, Methods and Future Directions (2018, methodology): “The same type of model was used by most of the 50 participating teams in the Fake News Challenge (Pomerleau and Rao, 2017), including most of the top entries (Riedel et al., 2017; Hanselowski et al., 2017).”
Detecting Fake News in Social Media Networks (2018, methodology): “Natural Language Processing (NLP) techniques have been used for news outlet stance detection to facilitate fake news detection on certain issues [20].”
Fake News Detection Using Machine Learning Ensemble Methods (2020, methodology): “A competitive solution is provided by Riedel et al. [19], which is a stance detection system that assigns one of four labels to an article, “agree,” “disagree,” “discuss,” or “unrelated,” depending on the conformity of article headline with article text. authors used linguistic properties of text…”
TwoWingOS: A Two-Wing Optimization Strategy for Evidential Claim Verification (2018, methodology): “…hidden layer, based on TF-IDF cosine similarity between the claim and the evidence (all evidence sentences are concatenated as a longer text piece) (Riedel et al., 2017) , the upper bound of accuracy for three classes if the coverage x % satisﬁes. tion mechanisms to decompose the problem into…”