Limits of Detecting Text Generated by Large-Scale Language Models
By L. Varshney, Nitish Shirish Keskar, Richard Socher. 2020
23 citations recorded by the source corpus.
Some consider large-scale language models that can generate long and coherent pieces of text as dangerous, since they may be used in misinformation campaigns. Here we formulate large-scale language model output detection as a hypothesis testing problem to classify text as genuine or generated. We show that error exponents for particular language models are bounded in terms of their perplexity, a standard measure of language generation performance. Under the assumption that human language is stationary and ergodic, the formulation is ex-tended from considering specific language models to considering maximum likelihood language models, among the class of k-order Markov approximations; error probabilities are characterized. Some discussion of incorporating semantic side information is also given.
Subfields: Artificial Intelligence, Sociology and Political Science, Large Language Models, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Benjamin Schiller
- Johannes Daxenberger
- Iryna Gurevych
- Jonas Pfeiffer
- Ivan Vulic
- Mohit Bansal
- Andreas Ruckl'e
- Sebastian Ruder
In Sociology and Political Science
In Large Language Models
In Safety, Robustness & Interpretability
Citations in the corpus
- Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text (2024, unknown): “From a theoretical perspective, Varshney et al. (2020), Helm et al. (2023), and Sadasivan et al. (2023) all discuss the limits of detection.”
- Aspect-Controlled Neural Argument Generation (2020, background): “Moreover, we believe that providing access to these models is of major importance, as their development cannot be regulated and, if they are open-sourced, it also encourages the work on counter-measures to detect them (Varshney et al., 2020).”