All the News That’s Fit to Fabricate: AI-Generated Text as a Tool of Media Misinformation

By Sarah Kreps, Miles McCain, Miles Brundage. 2020

349 citations recorded by the source corpus.

Abstract Online misinformation has become a constant; only the way actors create and distribute that information is changing. Advances in artificial intelligence (AI) such as GPT-2 mean that actors can now synthetically generate text in ways that mimic the style and substance of human-created news stories. We carried out three original experiments to study whether these AI-generated texts are credible and can influence opinions on foreign policy. The first evaluated human perceptions of AI-generated text relative to an original story. The second investigated the interaction between partisanship and AI-generated news. The third examined the distributions of perceived credibility across different AI model sizes. We find that individuals are largely incapable of distinguishing between AI- and human-generated text; partisanship affects the perceived credibility of the story; and exposure to the text does little to change individuals’ policy views. The findings have important implications in understanding AI in online misinformation campaigns.

Subfields: Communication, Sociology and Political Science, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Communication

In Sociology and Political Science

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Automatic Detection of Generated Text is Easiest when Humans are Fooled (2019, background): “Kreps et al. (2020), Gehrmann et al. (2019), and others have stressed the importance of humans being able to identify fake content on the web.”
TruthfulQA: Measuring How Models Mimic Human Falsehoods (2021, background): “Truthfulness is relevant to many applications including generating news stories (Kreps et al., 2020; Zellers et al., 2019), summarization (Gabriel et al., 2021; Maynez et al., 2020; Stiennon et al., 2020; Wang et al., 2020), conversational dialog (Shuster et al., 2021; Roller et al., 2021), and…”
The Gradient of Generative AI Release: Methods and Considerations (2023, background): “This is particularly important when models are deployed in high-stakes settings [47].”
Teaching Small Language Models to Reason (2022, background): “The concerns here are that both the teacher and student model may potentially generate non-factual (Ji et al., 2022; Pagnoni et al., 2021; Kreps et al., 2022) or offensive output (Gehman et al., 2020).”
Jailbroken: How Does LLM Safety Training Fail? (2023, background): “, for misinformation or for crime [9, 32, 25, 30, 28]).”
Generative AI (2023, neutral): “, fake news, propaganda) that is hard to detect by humans (Jakesch et al., 2023; Kreps et al., 2022).”
GPT-3 and InstructGPT: technological dystopianism, utopianism, and “Contextual” perspectives in AI ethics and industry (2022, background): “This could facilitate an increase in mis-information, such as false or misleading media, or augment the “coordinated hyper-targeting of articles to individual groups” (Kreps et al. [35], p. 2) with potentially serious consequences for radicalisation.”
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap (2023, background): “They can contribute—both intentionally and unintentionally—to the spread of misinfor-mation (Buchanan et al., 2021; Kreps et al., 2022; Zhou et al., 2023).”
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation (2021, result): “[37] shows that GPT-2 [38] synthetic text samples can achieve a promising convincing score compared with authentic articles from the New York Times (72% vs.”
Synthetic Lies: Understanding AI-Generated Misinformation and Evaluating Algorithmic and Human Solutions (2023, methodology): “For our work, we utilize this feature of LLMs as a generative approach, a technique that has been adopted in recent research for compiling datasets on news stories [59] and greeting card messages [100].”
Co-Writing with Opinionated Language Models Affects Users’ Views (2023, background): “However, more cautious voices have also warned about the ethical and social risks of harm from large language models [100, 101], ranging from discrimination and exclusion [23, 54, 83] to misinformation [67, 75, 91, 106] and environmental [96] and socioeconomic harms [14].”
Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers (2023, neutral): “These models generate coherent and ﬂ uent output, that can be dif ﬁ cult to distinguish from text written by humans 16,17 .”
Generative artificial intelligence (2023, neutral): “…GAI models provides starting points for new applications and business models for misuse and criminals, ultimately being leveraged to spread misinformation and influence the media and politics, or to defraud individuals and businesses (Hart-mann et al., 2023; Kreps et al., 2022; Mirsky & Lee, 2022).”