SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles

By Giovanni Da San Martino, Alberto Barrón-Cedeño, Henning Wachsmuth, Rostislav Petrov, Preslav Nakov. 2020

283 citations recorded by the source corpus.

We present the results and the main findings of SemEval-2020 Task 11 on Detection of Propaganda Techniques in News Articles. The task featured two subtasks. Subtask SI is about Span Identification: given a plain-text document, spot the specific text fragments containing propaganda. Subtask TC is about Technique Classification: given a specific text fragment, in the context of a full document, determine the propaganda technique it uses, choosing from an inventory of 14 possible propaganda techniques. The task attracted a large number of participants: 250 teams signed up to participate and 44 made a submission on the test set. In this paper, we present the task, analyze the results, and discuss the system submissions and the methods they used. For both subtasks, the best systems used pre-trained Transformers and ensembles.

Subfields: Artificial Intelligence, Sociology and Political Science, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yulia Tsvetkov
2. Vidhisha Balachandran
3. Sachin Kumar
4. Antonios Anastasopoulos
5. Lucille Njoo
6. John Lawrence
7. Chris Reed
8. Mohit Bansal

In Sociology and Political Science

1. Yulia Tsvetkov
2. Chris Reed

In Natural Language Processing

1. Yulia Tsvetkov
2. Sachin Kumar
3. Vidhisha Balachandran
4. Antonios Anastasopoulos
5. Lucille Njoo
6. Chris Reed
7. John Lawrence
8. Anjalie Field

In Evaluation & Benchmarks

1. Yulia Tsvetkov
2. Sachin Kumar
3. Vidhisha Balachandran
4. Chris Reed
5. John Lawrence
6. Antonios Anastasopoulos

Citations in the corpus

• Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “directions include automated fact-checking, propaganda, or fake news detection for which several annotated datasets (Oshikawa et al., 2020; Martino et al., 2020; Zhou and Zafarani, 2020; Guo et al., 2022; Huang et al., 2022) and shared tasks (Thorne et al.”
• Argument Mining: A Survey (2020, background): “The task requires identification of a wide range of phenomena including logical fallacies,43 techniques appealing to emotions, loaded language, and more (San Martino et al. 2019).”