A Stylometric Inquiry into Hyperpartisan and Fake News

By Martin Potthast, Johannes Kiesel, K. Reinartz, Janek Bevendorff, Benno Stein. Annual Meeting of the Association for Computational Linguistics, 2017

696 citations recorded by the source corpus.

We report on a comparative style analysis of hyperpartisan (extremely one-sided) news and fake news. A corpus of 1,627 articles from 9 political publishers, three each from the mainstream, the hyperpartisan left, and the hyperpartisan right, have been fact-checked by professional journalists at BuzzFeed: 97% of the 299 fake news articles identified are also hyperpartisan. We show how a style analysis can distinguish hyperpartisan news from the mainstream (F1 = 0.78), and satire from both (F1 = 0.81). But stylometry is no silver bullet as style-based fake news detection does not work (F1 = 0.46). We further reveal that left-wing and right-wing news share significantly more stylistic similarities than either does with the mainstream. This result is robust: it has been confirmed by three different modeling approaches, one of which employs Unmasking in a novel way. Applications of our results include partisanship detection and pre-screening for semi-automatic fake news detection.

Subfields: Artificial Intelligence, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/ed31e1225f6a76b469dfe4d022b235dc70be4390

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

Citations in the corpus

A Survey of Fake News (2020, methodology): “…4), which are initially developed for identifying deception in computer-mediated communications [Fuller et al. 2009; Zhou et al. 2004b] and testimonies [Afroz et al. 2012], and recently used in fake news detection [Bond et al. 2017; Pérez-Rosas et al. 2017; Potthast et al. 2017; Zhou et al. 2019a].”
SemEval-2019 Task 4: Hyperpartisan News Detection (2019, methodology): “We deliberately disregard the distinction between left and right, since previous work has found that, in hyperpartisan form, both are more similar to each other in terms of style than either are to the mainstream (Potthast et al., 2018c).”
GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social Media (2020, background): “…are investigated for supervised learning, including TF-IDF and topic features (Castillo et al., 2011), language styles (e.g., part of speech, factive/assertive verbs, and subjectivity) (Popat, 2017), writing styles and consistency (Potthast et al., 2018), and social emotions (Guo et al., 2019).”
Automatic Detection of Fake News (2017, methodology): “Third, recently, a stylometric (i.e. writing-style) approach has been proposed for the identiﬁcation of fake and genuine news articles (Potthast et al., 2017).”
Fake News: A Survey of Research, Detection Methods, and Opportunities (2018, background): “ed phenomena that are potentially useful. Feature selection for such engineered features is often time-consuming and labor-intensive. Related fake news studies can be seen in, e.g., [Bond et al. 2017;Potthast et al. 2017;Volkova et al. 2017]. When automatically learning features, deep learning techn”
Fake News Detection on Social Media: A Data Mining Perspective (2017, background): “(KNN), and support vector machines (SVM), and then selecting the classifier that performs the best [62; 75; 1].”
Where is Your Evidence: Improving Fact-checking by Justification Modeling (2018, background): “Thorne and Vlachos (2018) provide a survey of current datasets and models for factchecking (e.g., (Wang, 2017; Rashkin et al., 2017; Vlachos and Riedel, 2014; Thorne et al., 2018; Long et al., 2017; Potthast et al., 2018; Wang et al., 2018)).”
Evaluating adversarial attacks against multiple fact verification systems (2019, background): “, 2018a) or biased news detection (Potthast et al., 2018), it is critical to understand how systems and models behave when exposed to real-world data and how deficiencies in their training data may contribute to this.”
SpotFake: A Multi-modal Framework for Fake News Detection (2019, background): “[7] showed how writing style, network connection and user reaction can lead to the detection of fake news.”
Fake News Detection on Social Media using Geometric Deep Learning (2019, background): “Content-based approaches, which are used in the majority of works on fake news detection, rely on linguistic (lexical and syntactical) features that can capture deceptive cues or writing styles [1, 32, 30, 29, 28].”
Returning the N to NLP: Towards Contextually Personalized Classification Models (2020, background): “Other explored tasks include topic detection, part-of-speech tagging (Hovy, 2015), prepositional phrase attachment, sarcasm detection (Lynn et al., 2017), fake news detection (Long et al., 2017; Potthast et al., 2018), or detection of mental health issues (Benton et al., 2016).”
Preserving integrity in online social networks (2020, background): “[99] analyze the writing style of hyper-partisan (i.”
DialFact: A Benchmark for Fact-Checking in Dialogue (2021, background): “, 2020), news articles (Potthast et al., 2018; Shu et al., 2020), and social media claims (Nakov et al.”