Inducing Multilingual Text Analysis Tools via Robust Projection across Aligned Corpora

By David Yarowsky, G. Ngai, R. Wicentowski. Human Language Technology - The Baltic Perspectiv, 2001

630 citations recorded by the source corpus.

This paper describes a system and set of algorithms for automatically inducing stand-alone monolingual part-of-speech taggers, base noun-phrase bracketers, named-entity taggers and morphological analyzers for an arbitrary foreign language. Case studies include French, Chinese, Czech and Spanish.Existing text analysis tools for English are applied to bilingual text corpora and their output projected onto the second language via statistically derived word alignments. Simple direct annotation projection is quite noisy, however, even with optimal alignments. Thus this paper presents noise-robust tagger, bracketer and lemmatizer training procedures capable of accurate system bootstrapping from noisy and incomplete initial projections.Performance of the induced stand-alone part-of-speech tagger applied to French achieves 96% core part-of-speech (POS) tag accuracy, and the corresponding induced noun-phrase bracketer exceeds 91% F-measure. The induced morphological analyzer achieves over 99% lemmatization accuracy on the complete French verbal system.This achievement is particularly noteworthy in that it required absolutely no hand-annotated training data in the given language, and virtually no language-specific knowledge or resources beyond raw text. Performance also significantly exceeds that obtained by direct annotation projection.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/0ce9c261a7f69668da2066da0ad736e6eccdcd36

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Bootstrapping parsers via syntactic projection across parallel texts (2005, methodology): “Annotation projection using parallel text has been accomplished for shallower tasks (Yarowsky and Ngai 2001; Merlo, Stevenson, Tsang and Allaria 2002; Yarowsky, Ngai and Wicentowski 2001), but the projection of tree structures introduces additional complexity.”
Unsupervised Multilingual Learning for Morphological Segmentation (2008, background): “Multilingual Language Learning Recently, the availability of parallel corpora has spurred research on multilingual analysis for a variety of tasks ranging from morphology to semantic role labeling (Yarowsky et al., 2000; Diab and Resnik, 2002; Xi and Hwa, 2005; Padó and Lapata, 2006).”
Generating High Quality Proposition Banks for Multilingual Semantic Role Labeling (2015, background): “Annotation Projection in Parallel Corpora to train monolingual tools for new languages was introduced in the context of learning a PoS tagger (Yarowsky et al., 2001).”
Unsupervised models for morpheme segmentation and morphology learning (2007, methodology): “However, espe cially with large amounts of data Linguistica is oversensitive to common strings that oc cur at the end of words and proposes segmentations, such as ‘allu+de, alongsi+de, lon gitu+de’; ‘anyh+ow, highbr+ow, longfell+ow’. Solutions to this problem could be found, e.g., in the approa ch a”
Ten Pairs to Tag – Multilingual POS Tagging via Coarse Mapping between Embeddings (2016, methodology): “Multilingual POS Tagging Prior work on multilingual POS tagging has mainly focused on the tag projection method (Yarowsky et al., 2001; Wisniewski et al., 2014; Duong et al., 2013; Duong et al., 2014; Täckström et al., 2013; Das and Petrov, 2011; Snyder et al., 2008; Naseem et al., 2009; Chen et al.”
Learning multilingual named entity recognition from Wikipedia (2013, background): “Given a high-performance ner system, phrase-aligned corpora and machine translation may enable the transference of ne knowledge from well-resourced languages to others [89,64,69,39,28,21].”
MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages (2022, methodology): “Early cross-lingual and multilingual NLU modeling approaches used projection or alignment methods (Yarowsky et al., 2001), focusing on string matching, edit distance, or consonant signatures (Ehrmann et al., 2011), lookup lexicons for lowresource languages (Mayhew et al., 2017), and aligning (Xie…”
ParaCrawl: Web-Scale Acquisition of Parallel Corpora (2020, background): “, 2019) or cross-lingual projection of language tools (Yarowsky et al., 2001).”
SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings (2020, methodology): “…have been successfully used in a range of tasks such as ty-pological analysis (Lewis and Xia, 2008; ¨Ostling, 2015b), annotation projection (Yarowsky et al., 2001; Hwa et al., 2002; Pad´o and Lapata, 2009) and creating multilingual embeddings (Guo et al., 2016; Ammar et al., 2016; Dufter…”
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios (2020, background): “These labels (on the high-resource text) can then be projected back to the text in the low-resource language based on the alignment between tokens in the parallel texts (Yarowsky et al., 2001).”
JW300: A Wide-Coverage Parallel Corpus for Low-Resource Languages (2019, background): “The formative work by Yarowsky et al. (2001) has inspired many inﬂuential works in projecting sequential labels from multiple source languages (Das and Petrov, 2011; T ¨ ackstr ¨ om et al., 2013), as well as projecting more complex annotations such as syntactic and semantic dependencies (Hwa et…”
Cross-lingual Transfer Learning for Multilingual Task Oriented Dialog (2018, methodology): “The slot annotations are projected via the attention weights (Yarowsky et al., 2001).”
A Systematic Comparison of Various Statistical Alignment Models (2003, background): “In Yarowsky, Ngai, and Wicentowski (2001), word alignment is used to transfer text analysis tools such as morphologic analyzers or part-of-speech taggers from a language, such as English, for which many tools already exist to languages for which such resources are scarce.”
Massively Multilingual Transfer for NER (2019, methodology): “…high-to low-resource languages, which has typically taken the form of annotation projection over parallel corpora or other multilingual resources (Yarowsky et al., 2001; Hwa et al., 2005), or making use of trans-ferable representations, such as phonetic transcriptions (Bharadwaj et al., 2016),…”
Challenges and Strategies in Cross-Cultural NLP (2022, methodology): “Using parallel data and a word alignment tool, annotation (for example a syntactic tree) in a source language can be transferred to a target language without extra annotation (Yarowsky et al., 2001; Hwa et al., 2005).”
The Web as a Parallel Corpus (2003, background): “…as the bridge, and then use robust statistical techniques in learning from the resulting noisy annotations (Cabezas, Dorr, and Resnik 2001; Diab and Resnik 2002; Hwa et al. 2002; Lopez et al. 2002; Yarowsky, Ngai, and Wicentowski 2001; Yarowsky and Ngai 2001; Riloff, Schafer, and Yarowsky 2002).”
Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks (2016, background): “Resource-based transfer utilizes additional linguistic annotations as weak supervision for transfer learning, such as cross-lingual dictionaries (Zirikly & Hagiwara, 2015), corpora (Wang & Manning, 2014), and word alignments (Yarowsky et al., 2001).”
Many Languages, One Parser (2016, background): “Another popular approach for cross-lingual supervision is to project annotations from the source language to the target language via a parallel corpus (Yarowsky et al., 2001; Hwa et al., 2005) or via automatically-translated sentences (Tiedemann et al.”