Learning multilingual named entity recognition from Wikipedia

By Joel Nothman, Nicky Ringland, Will Radford, T. Murphy, James Richard Curran. Artificial Intelligence, 2013

401 citations recorded by the source corpus.

We automatically create enormous, free and multilingual silver-standard training annotations for named entity recognition (ner) by exploiting the text and structure of Wikipedia. Most ner systems rely on statistical models of annotated data to identify and classify names of people, locations and organisations in text. This dependence on expensive annotation is the knowledge bottleneck our work overcomes. We first classify each Wikipedia article into named entity (ne) types, training and evaluating on 7200 manually-labelled Wikipedia articles across nine languages. Our cross-lingual approach achieves up to 95% accuracy. We transform the links between articles into ne annotations by projecting the target [email protected]?s classifications onto the anchor text. This approach yields reasonable annotations, but does not immediately compete with existing gold-standard data. By inferring additional links and heuristically tweaking the Wikipedia corpora, we better align our automatic annotations to gold standards. We annotate millions of words in nine languages, evaluating English, German, Spanish, Dutch and Russian Wikipedia-trained models against conll shared task data and other gold-standard corpora. Our approach outperforms other approaches to automatic ne annotation (Richman and Schone, 2008 [61], Mika et al., 2008 [46]) competes with gold-standard training when tested on an evaluation corpus from a different source; and performs 10% better than newswire-trained models on manually-annotated Wikipedia text.

Subfields: Artificial Intelligence, Communication

View Paper: https://www.semanticscholar.org/paper/696b505083d34c6f995aef88d0352d70d7f7e8c8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

Citations in the corpus

A Joint Model for Entity Analysis: Coreference, Typing, and Linking (2014, methodology): “We use three sources of semantics from Wikipedia (Kazama and Torisawa, 2007; Nothman et al., 2013):”
Cross-lingual Name Tagging and Linking for 282 Languages (2017, methodology): “…based silver standard generation: Our work was mainly inspired from previous work that leveraged Wikipedia markups to train name taggers (Nothman et al., 2008; Dakka and Cucerzan, 2008; Mika et al., 2008; Ringland et al., 2009; Alotaibi and Lee, 2012; Nothman et al., 2013; Althobaiti et al., 2014).”
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages (2020, methodology): “For NER, we train and evaluate Sta n z a with 12 publicly available datasets covering 8 major languages as shown in Table 3 (Nothman et al., 2013; Tjong Kim Sang and De Meulder, 2003; Tjong Kim Sang, 2002; Benikova et al., 2014; Mohit et al., 2012; Taulé et al., 2008; Weischedel et al., 2013).”
Massively Multilingual Transfer for NER (2019, methodology): “The resulting cross-lingual embeddings are then used in NER direct transfer in a leave-one-out setting for the 41 languages ( 41 × 40 transfers), and we report the mean F 1 in Table 3.”
MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages (2022, background): “…differences between samples in different languages, because they are natural reﬂections of the questions that programmers asked in each speciﬁc language, similar to many works in ﬁelds such as multilingual question answering (Clark et al., 2020) and named entity recognition (Nothman et al., 2013).”
Multilingual Language Processing From Bytes (2015, positive): “Nothman et al. (2013) use Wikipedia anchor links and disambiguation pages joined with Freebase types to create a huge amount of somewhat noisy training data and are able to achieve good results on many languages (with some extra heuristics).”
FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP (2019, background): “, IMDB (Maas et al., 2011), TREC-6 (Voorhees and Harman, 2000), TREC-50 (Li and Roth, 2002), Universal De-pendencies (Zeman et al., 2018), WikiNER (Nothman et al., 2012) and WNUT-17 (Derczynski et al., 2017). treebank for English, simply execute these lines: # define dataset task = NLPTask.”
DyNet: The Dynamic Neural Network Toolkit (2017, methodology): “Models were trained and tested on the WikiNER English Corpus [48], and all words with frequency less than ﬁve were treated as unknowns.”
IXA pipeline: Efficient and Ready to Use Multilingual NLP tools (2014, methodology): “Both Perceptron (Collins, 2002; Collins, 2003; Ratinov and Roth, 2009) and Maximum Entropy models (Ratnaparkhi and others, 1996; Charniak, 2000; Clark and Curran, 2003; Toutanova et al., 2003; Nothman et al., 2012) are adaptable algorithms which have largely been applied to NLP tasks such as POS tag”
Toward Mention Detection Robustness with Recurrent Neural Networks (2016, neutral): “The state-of-the-art performance for this dataset is due to Carreras et al. (2002) in the CoNLL 2002 evaluation and Nothman et al. (2013).”
Knowledge graph refinement: A survey of approaches and evaluation methods (2016, methodology): “However, assuming that the given knowledge graph is already of reasonable quality, we call this method silver standard evaluation, as already proposed in other works [32,44,73].”