Faking Fake News for Real Fake News Detection: Propaganda-Loaded Training Data Generation

By Kung-Hsiang Huang, Preslav Nakov, Yejin Choi, Heng Ji. 2022

80 citations recorded by the source corpus.

Despite recent advances in detecting fake news generated by neural models, their results are not readily applicable to effective detection of human-written disinformation. What limits the successful transfer between them is the sizable gap between machine-generated fake news and human-authored ones, including the notable differences in terms of style and underlying intent. With this in mind, we propose a novel framework for generating training examples that are informed by the known styles and strategies of human-authored propaganda. Specifically, we perform self-critical sequence training guided by natural language inference to ensure the validity of the generated articles, while also incorporating propaganda techniques, such as appeal to authority and loaded language. In particular, we create a new training dataset, PropaNews, with 2,256 examples, which we release for future use. Our experimental results show that fake news detectors trained on PropaNews are better at detecting human-written disinformation by 3.62–7.69% F1 score on two public datasets.

Subfields: Artificial Intelligence, Sociology and Political Science, Natural Language Processing, Evaluation & Benchmarks, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yulia Tsvetkov
2. Vidhisha Balachandran
3. Sachin Kumar
4. Antonios Anastasopoulos
5. Lucille Njoo
6. Mohit Bansal
7. Anjalie Field
8. Hinrich Schütze

In Sociology and Political Science

1. Yulia Tsvetkov

In Natural Language Processing

1. Yulia Tsvetkov
2. Sachin Kumar
3. Vidhisha Balachandran
4. Antonios Anastasopoulos
5. Lucille Njoo
6. Anjalie Field
7. Hinrich Schütze
8. Mohit Bansal

In Evaluation & Benchmarks

1. Yulia Tsvetkov
2. Sachin Kumar
3. Vidhisha Balachandran
4. Antonios Anastasopoulos

In Signal Processing

1. Vidhisha Balachandran
2. Yulia Tsvetkov
3. Antonios Anastasopoulos
4. Sachin Kumar

Citations in the corpus

• Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “directions include automated fact-checking, propaganda, or fake news detection for which several annotated datasets (Oshikawa et al., 2020; Martino et al., 2020; Zhou and Zafarani, 2020; Guo et al., 2022; Huang et al., 2022) and shared tasks (Thorne et al.”