How Robust Are Character-Based Word Embeddings in Tagging and MT Against Wrod Scramlbing or Randdm Nouse?

By Georg Heigold, Günter Neumann, Josef van Genabith. 2017

67 citations recorded by the source corpus.

This paper investigates the robustness of NLP against perturbed word forms. While neural approaches can achieve (almost) human-like accuracy for certain tasks and conditions, they often are sensitive to small changes in the input such as non-canonical input (e.g., typos). Yet both stability and robustness are desired properties in applications involving user-generated content, and the more as humans easily cope with such noisy or adversary conditions. In this paper, we study the impact of noisy input. We consider different noise distributions (one type of noise, combination of noise types) and mismatched noise distributions for training and testing. Moreover, we empirically evaluate the robustness of different models (convolutional neural networks, recurrent neural networks, non-neural models), different basic units (characters, byte pair encoding units), and different NLP tasks (morphological tagging, machine translation).

Subfields: Natural Language Processing, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Machine Translation

Citations in the corpus

Analysis Methods in Neural Language Processing: A Survey (2018, background): “There is relatively little work on adversarial examples for more low-level language processing tasks, although one can mention morphological tagging (Heigold et al., 2018) and spelling correction (Sakaguchi et al., 2017).”
On internal language representations in deep learning: an analysis of machine translation and speech recognition (2018, background): “Neural machine translation models are also very sensitive to input noise, such as character-level transformation [26, 146].”
Evaluating the Robustness of Neural Language Models to Input Perturbations (2021, background): “Almost all the character-level perturbations presented here were already tested in adversarial attack scenarios (Heigold et al., 2018; Zeng et al., 2020; Zhang et al., 2020), but were not yet implemented in a non-adversarial testing framework, except the misspelling perturbation implemented by…”
One Size Does Not Fit All: Comparing NMT Representations of Different Granularities (2019, background): “explore representations from robustly trained systems, which should improve performance on noisy input (Belinkov and Bisk, 2018; Heigold et al., 2018).”