Imitation Attacks and Defenses for Black-box Machine Translation Systems

By Eric Wallace, Mitchell Stern, Dawn Song. 2020

142 citations recorded by the source corpus.

We consider an adversary looking to steal or attack a black-box machine translation (MT) system, either for financial gain or to exploit model errors. We first show that black-box MT systems can be stolen by querying them with monolingual sentences and training models to imitate their outputs. Using simulated experiments, we demonstrate that MT model stealing is possible even when imitation models have different input data or architectures than their victims. Applying these ideas, we train imitation models that reach within 0.6 BLEU of three production MT systems on both high-resource and low-resource language pairs. We then leverage the similarity of our imitation models to transfer adversarial examples to the production systems. We use gradient-based attacks that expose inputs which lead to semantically-incorrect translations, dropped content, and vulgar model outputs. To mitigate these vulnerabilities, we propose a defense that modifies translation outputs in order to misdirect the optimization of imitation models. This defense degrades imitation model BLEU and attack transfer rates at some cost in BLEU and inference speed.

Subfields: Artificial Intelligence, Software, Natural Language Processing, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Software

In Natural Language Processing

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

The False Promise of Imitating Proprietary LLMs (2023, methodology): “In this work, we study one possible resolution to this question: model imitation (Wallace et al., 2020; Orekondy et al., 2019).”
FUDGE: Controlled Text Generation With Future Discriminators (2021, background): “We recognize that strong controlled generation methods have the potential to produce harmful outputs and/or misinformation when used adversarially (Wallace et al., 2019, 2020).”
Controlled Text Generation as Continuous Optimization with Multiple Constraints (2021, background): “Additionally, they often require several heuristics to work and are prone to adversarial outputs [64].”
Gradient-based Constrained Sampling from Language Models (2022, background): “They are prone to generating toxic (Gehman et al., 2020) and non-factual content (Pagnoni et al., 2021), and can potentially be used maliciously (Wallace et al., 2019, 2020; Zellers et al., 2019).”