Ten Pairs to Tag – Multilingual POS Tagging via Coarse Mapping between Embeddings

By Yuan Zhang, David Gaddy, Regina Barzilay, Tommi S. Jaakkola. 2016

124 citations recorded by the source corpus.

In the absence of annotations in the target language, multilingual models typically draw on extensive parallel resources. In this paper, we demonstrate that accurate multilingual part-of-speech (POS) tagging can be done with just a few (e.g., ten) word translation pairs. We use the translation pairs to establish a coarse linear isometric (orthonormal) mapping be-tween monolingual embeddings. This enables the supervised source model expressed in terms of embeddings to be used directly on the target language. We further reﬁne the model in an unsupervised manner by initializing and regularizing it to be close to the direct transfer model. Averaged across six languages, our model yields a 37.5% absolute improvement over the monolingual prototype-driven method (Haghighi and Klein, 2006) when using a comparable amount of super-vision. Moreover, to highlight key linguistic characteristics of the generated tags, we use them to predict typological properties of languages, obtaining a 50% error reduction relative to the prototype model. 1

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

XNLI: Evaluating Cross-lingual Sentence Representations (2018, background): “For instance, Zhang et al. (2016) show that cross-lingual embeddings can be used to extend an English part-of-speech tagger to the cross-lingual setting, and Xiao and Guo (2014) achieve similar results in dependency parsing.”
Learning bilingual word embeddings with (almost) no bilingual data (2017, methodology): “The same optimization objective is used by Zhang et al. (2016), who constrain the transformation matrix to be orthogonal.”
A Survey of Cross-lingual Word Embedding Models (2017, background): “Zhang et al. (2016b) furthermore map proper nouns to nouns and symbol makers (e.g. “-”, “/”) and interjections to an X tag as it is hard and unnecessary to disambiguate them in a low-resource setting.”
A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings (2018, background): “…map the embeddings in one or both languages under the constraint of the transformation being orthogonal (Xing et al., 2015; Artetxe et al., 2016; Zhang et al., 2016; Smith et al., 2017), and margin meth-ods, which map the embeddings in one language to maximize the margin between the correct…”
Exploring Fine-tuning Techniques for Pre-trained Cross-lingual Models via Continual Learning (2020, methodology): “…been applied to multiple NLP tasks, such as task-oriented dialogue systems (Liu et al., 2019a,b), part-of-speech tagging (Wisniewski et al., 2014; Zhang et al., 2016; Kim et al., 2017), named entity recognition (Ni et al., 2017; Xie et al., 2018), abstractive summarization (Duan et al., 2019;…”
Model Transfer for Tagging Low-resource Languages using a Bilingual Dictionary (2017, methodology): “Parallel data therefore appears to be the most realistic additional source of information for developing NLP systems in low-resource languages (Yarowsky and Ngai, 2001; Das and Petrov, 2011; Täckström et al., 2013; Fang and Cohn, 2016; Zhang et al., 2016).”
Should All Cross-Lingual Embeddings Speak English? (2019, methodology): “BLI is often the ﬁrst step towards several down-stream tasks such as Part-Of-Speech (POS) tagging (Zhang et al., 2016), parsing (Ammar et al., 2016a), document classiﬁcation (Klementiev et al., 2012), and machine translation (Irvine and Callison-Burch, 2013; Artetxe et al., 2018b; Lample et al.,…”
ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning (2023, background): “However, to our knowledge, none of existing work has conducted large-scale evaluations of ChatGPT for multiple and diverse languages/tasks as we do.”
Gromov-Wasserstein Alignment of Word Embedding Spaces (2018, methodology): “Steps beyond this constraint include using small amounts of parallel data (Zhang et al., 2016) or an unsupervised technique as the initial step to generate pseudo-parallel data (Conneau et al., 2018) before solving for P .”
Earth Mover’s Distance Minimization for Unsupervised Bilingual Lexicon Induction (2017, methodology): “With this in mind, we ﬁnd the earth mover’s distance to be a suitable choice (Zhang et al., 2016b).”