Adversarial Evaluation for Models of Natural Language

By Noah A. Smith. 2012

22 citations recorded by the source corpus.

We now have a rich and growing set of modeling tools and algorithms for inducing linguistic structure from text that is less than fully annotated. In this paper, we discuss some of the weaknesses of our current methodology. We present a new abstract framework for evaluating natural language processing (NLP) models in general and unsupervised NLP models in particular. The central idea is to make explicit certain adversarial roles among researchers, so that the different roles in an evaluation are more clearly defined and performers of all roles are offered ways to make measurable contributions to the larger goal. Adopting this approach may help to characterize model successes and failures by encouraging earlier consideration of error analysis. The framework can be instantiated in a variety of ways, simulating some familiar intrinsic and extrinsic evaluations as well as some new evaluations. This paper presents a new approach to evaluating computational models of natural language, based on adversarial roles performed by different researchers or their models. We begin inx2 with a brief review of current evaluation strategies in NLP. We then turn to coupled adversarial evaluations inspired by perplexity (x3) and the traditional roles of linguists (x4). The two-performer setup is formalized inx5. We then consider the origins of the data and growing awareness of the importance of context on language use (x6) and provide a three-performer setup, in which a third performer manages data selection (x7). We close with a few open questions (x8).

Subfields: Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Adversarial Examples for Evaluating Reading Comprehension Systems (2017, background): “Generative adversarial models pit a generative model, whose goal is to generate realistic outputs, against a discriminative model, whose goal is to distinguish the generator’s outputs from real data (Smith, 2012; Goodfellow et al., 2014).”
Stress Test Evaluation for Natural Language Inference (2018, background): “Smith (2012) discuss dangers of community-wide “overfitting” to benchmark datasets and emphasize the need to correlate model errors to well-defined linguistic phenomena to understand specific model strengths and weaknesses.”
The Enemy in Your Own Camp: How Well Can We Detect Statistically-Generated Fake Reviews – An Adversarial Study (2016, methodology): “Our contributions We set up an adversarial evaluation approach inspired by (Smith, 2012), using graphical models to build various language models that generate fake reviews, with and without recurrence to meta-information.”
Robust Training under Linguistic Adversity (2017, background): “Based on the same linguistic intuition, adversarial evaluation for natural language processing models was proposed by Smith (2012).”