Authorship Attribution for Neural Text Generation

By Adaku Uchendu, Thai Le, Kai Shu, Dongwon Lee. 2020

249 citations recorded by the source corpus.

In recent years, the task of generating realistic short and long texts have made tremendous advancements. In particular, several recently proposed neural network-based language models have demonstrated their astonishing capabilities to generate texts that are challenging to distinguish from human-written texts with the naked eye. Despite many benefits and utilities of such neural methods, in some applications, being able to tell the “author” of a text in question becomes critically important. In this work, in the context of this Turing Test, we investigate the so-called authorship attribution problem in three versions: (1) given two texts T1 and T2, are both generated by the same method or not? (2) is the given text T written by a human or machine? (3) given a text T and k candidate neural methods, can we single out the method (among k alternatives) that generated T? Against one humanwritten and eight machine-generated texts (i.e., CTRL, GPT, GPT2, GROVER, XLM, XLNET, PPLM, FAIR), we empirically experiment with the performance of various models in three problems. By and large, we find that most generators still generate texts significantly different from human-written ones, thereby making three problems easier to solve. However, the qualities of texts generated by GPT2, GROVER, and FAIR are better, often confusing machine classifiers in solving three problems. All codes and datasets of our experiments are available at: https://bit.ly/ 302zWdz

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Do Language Models Plagiarize? (2022, background): “A distinction between machine-authored and human-written content has even become quite challenging (Clark et al., 2021; Uchendu et al., 2020).”
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text (2023, background): “However, models trained explicitly to detect machine-generated texts may overfit their training distribution of the domains (Bakhtin et al., 2019; Uchendu et al., 2020).”
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature (2023, background): “However, Bakhtin et al. (2019); Uchendu et al. (2020) note that models trained explicitly to detect machine-generated text tend to overﬁt to their training distribution of data or source models.”
Automatic Detection of Machine Generated Text: A Critical Survey (2020, methodology): “Malicious usage : TGMs can have unfortunate uses by (even low-skilled) adversaries for malicious purposes, such as fake news generation (Zellers et al., 2019; Brown et al., 2020; Uchendu et al., 2020), fake product reviews generation (Adelani et al., 2020), and spamming/phishing (Weiss, 2019).”
Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature (2023, background): “…typically trained using bag-of-words (Solaiman et al., 2019; Fagni et al., 2021) or neural representations (Bakhtin et al., 2019; Solaiman et al., 2019; Uchendu et al., 2020; Ippolito et al., 2020; Fagni et al., 2021; Yan et al., 2023; Pu et al., 2023; Mitrovi ´ c et al., 2023; Li et al., 2023).”
A Statistical Turing Test for Generative Models (2023, methodology): “Indeed, detectors typically use information from the model used to generate the content to perform the detection [20, 30] and experience performance degradation when the generative model is not known a priori and available or the content is otherwise out of domain [3, 42, 27].”