Many Languages, One Parser

By Bridger Waleed Ammar, George Mulcaire, Miguel Ballesteros, Chris Dyer, Noah A. Smith. 2016

233 citations recorded by the source corpus.

We train one multilingual model for dependency parsing and use it to parse sentences in several languages. The parsing model uses (i) multilingual word clusters and embeddings; (ii) token-level language information; and (iii) language-specific features (fine-grained POS tags). This input representation enables the parser not only to parse effectively in multiple languages, but also to generalize across languages based on linguistic universals and typological similarities, making it more effective to learn from limited annotations. Our parser’s performance compares favorably to strong baselines in a range of data scenarios, including when the target language has a large treebank, a small treebank, or no treebank for training.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

A Survey of Cross-lingual Word Embedding Models (2017, methodology): “Despite its obvious simplicity, the only work that generalizes pseudo-bilingual approaches to the multilingual setting that we are aware of is by Ammar et al. (2016b) who replace all tokens in monolingual corpora with their corresponding translation cluster ID, thereby restricting them to have the…”
Learning Language Representations for Typology Prediction (2017, methodology): “, 2016), such as multilingual dependency parsing (Ammar et al., 2016), generative parsing in low-resource”
Exploiting Multi-typed Treebanks for Parsing with Deep Multi-task Learning (2016, methodology): “Unlike Duong et al. (2015b) and Ammar et al. (2016), we don’t use external resources such as cross-lingual word clusters, embeddings and dictionaries which is beyond the scope of this work.”
From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers (2020, background): “The previous state-of-the-art, cross-lingual word embeddings (CLWEs) (Mikolov et al., 2013; Ammar et al., 2016; Artetxe et al., 2017; Smith et al., 2017; Glavaˇs et al., 2019; Vuli´c et al., 2019) and sentence embeddings (Artetxe and Schwenk, 2019), have most recently been replaced by massively…”
Choosing Transfer Languages for Cross-Lingual Learning (2019, methodology): “There are many methods for performing this transfer, including joint training (Ammar et al., 2016; Tsai and Roth, 2016; Cotterell and Heigold, 2017; Johnson et al., 2017; Malaviya et al., 2018), annotation projection (T¨ackstr¨om et al., 2012; T¨ackstr¨om et al., 2013; Zhang et al., 2016; Ponti et…”
Massively Multilingual Transfer for NER (2019, methodology): “…transfer in NER (T¨ackstr¨om, 2012; Fang et al., 2017; Enghoff et al., 2018), POS tagging (Snyder et al., 2009; Plank and Agi´c, 2018), and parsing (Ammar et al., 2016) compared to single source transfer, however, multi-source transfer might be noisy as a result of divergence in script, phonology,…”
Balancing Training for Multilingual Neural Machine Translation (2020, background): “, 2018a), syntactic analysis (Plank et al., 2016; Ammar et al., 2016), named-entity recognition (Xie et al.”
Should All Cross-Lingual Embeddings Speak English? (2019, methodology): “BLI is often the ﬁrst step towards several down-stream tasks such as Part-Of-Speech (POS) tagging (Zhang et al., 2016), parsing (Ammar et al., 2016a), document classiﬁcation (Klementiev et al., 2012), and machine translation (Irvine and Callison-Burch, 2013; Artetxe et al., 2018b; Lample et al.,…”
Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning (2016, methodology): “Independently, Ammar et al. (2016) used a different polyglot architecture for multilingual dependency parsing.”
Edinburgh Research Explorer A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages (unknown): “…have not exhausted all the possible techniques (e.g., use of external resources (Ra-sooli and Collins, 2017; Rosa and Mareˇcek, 2018), predicted POS (Ammar et al., 2016), multiple source treebanks (Lim et al., 2018; Stymne et al., 2018), among others), we show that simple meth-ods which leverage…”
Ranking Transfer Languages with Pragmatically-Motivated Features for Multilingual Sentiment Analysis (2020, background): “…natural language processing tasks, including machine translation (Zoph et al., 2016; Johnson et al., 2017; Neubig and Hu, 2018), multilingual dependency parsing (Ammar et al., 2016; Ponti et al., 2018), and multilingual sentiment analysis (Prettenhofer and Stein, 2010; Zhou et al., 2016).”
UDapter: Language Adaptation for Truly Universal Dependency Parsing (2020, methodology): “Considering this, multilingual models leveraging Universal Dependency annotations (Nivre et al., 2018) have drawn serious attention from the NLP community (Zhang and Barzilay, 2015; Ammar et al., 2016; de Lhoneux et al., 2018; Kondratyuk and Straka, 2019).”
Polyglot Contextual Representations Improve Crosslingual Transfer (2019, background): “Previous work has shown this to be true to some degree in the context of syntactic dependency parsing (Ammar et al., 2016), semantic role labeling (Mulcaire et al., 2018), named entity recognition (Xie et al., 2018), and language modeling for phonetic sequences (Tsvetkov et al., 2016) and for…”
Multilingual Neural Machine Translation with Task-Specific Attention (2018, background): “…while the rest of the parameters are task-speciﬁc; Zoph and Knight (2016) jointly train the encoders while the rest of the parameters are task-speciﬁc, and Johnson et al. (2017) train both encoders and decoders jointly with language-speciﬁc tokens to guide learning as in (Ammar et al., 2016).”
URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors (2017, background): “It does not take long in collecting a “polyglot” experiment like those in Ammar et al. (2016), Tsvetkov et al. (2016), or Daiber et al. (2016) before one adds a language for which an expected feature is missing, present only in another database or not present in any database; this problem is…”
Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints (2017, background): “…2013; Devlin et al., 2014), dependency and semantic parsing (Socher et al., 2013a; Bansal et al., 2014; Chen and Manning, 2014; Johannsen et al., 2015; Ammar et al., 2016), sentiment analysis (Socher et al., 2013b), named entity recognition (Turian et al., 2010; Guo et al., 2014), and many others.”