A Cross-Lingual Dictionary for English Wikipedia Concepts

By Valentin I. Spitkovsky, Angel X. Chang. 2012

209 citations recorded by the source corpus.

We present a resource for automatically associating strings of text with English Wikipedia concepts. Our machinery is bi-directional, in the sense that it uses the same fundamental probabilistic methods to map strings to empirical distributions over Wikipedia articles as it does to map article URLs to distributions over short, language-independent strings of natural language text. For maximal inter-operability, we release our resource as a set of flat line-based text files, lexicographically sorted and encoded with UTF-8. These files capture joint probability distributions underlying concepts (we use the terms article, concept and Wikipedia URL interchangeably) and associated snippets of text, as well as other features that can come in handy when working with Wikipedia articles and related information.

Subfields: Artificial Intelligence, Communication, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Natural Language Processing

Citations in the corpus

Knowledge Enhanced Contextual Word Representations (2019, methodology): “The candidate selectors and priors are a combination of CrossWikis, a large, precomputed dictionary that combines statistics from Wikipedia and a web corpus (Spitkovsky and Chang, 2012), and the YAGO dictionary (Hoffart et al., 2011).”
Design Challenges for Entity Linking (2015, methodology): “Furthermore, we carry out an extensive ablation analysis, whose results illustrate 1) even a near-trivial model using CrossWikis (Spitkovsky and Chang, 2012) performs surprisingly well, and 2) incorporating a fine-grained set of entity types raises that level even higher.”
Natural Language Questions for the Web of Data (2012, background): “Spitkovsky and Chang (2012) recently released a huge dictionary of pairs of phrases and Wikipedia links, derived from Google’s Web index.”
Learning Text Representations for 500K Classification Tasks on Named Entity Disambiguation (2018, methodology): “We ﬁrst built a candidate model as a dictionary that links each text anchor to possible entities, using the method presented in (Spitkovsky and Chang, 2012; Barrena et al., 2016).”
Language Models are Open Knowledge Graphs (2020, methodology): “Entity linking to KG schema We leverage an unsupervised entity linker based on a mentionto-entity dictionary (Spitkovsky & Chang, 2012) to link the entities for scalability consideration.”
Entity query feature expansion using knowledge base links (2014, methodology): “For each Wikipedia entity we extract an entity representation consisting of its article text, canonical name, categories, and a distribution over aliases from redirects, Wikipedia-internal anchor text, and web anchor text from the Google cross-lingual dictionary [33].”
REL: An Entity Linker Standing on the Shoulders of Giants (2020, methodology): “To compute this prior, we sum up hyperlink counts from Wikipedia and from the CrossWikis corpus [25] to estimate probability PWiki(e |m).”
Fine-Grained Entity Typing for Domain Independent Entity Linking (2019, background): “, Gupta, Singh, and Roth (2017) use CrossWikis (Spitkovsky and Chang 2012) and re-”
Entity Linking via Joint Encoding of Types, Descriptions, and Context (2017, methodology): “For each mention m , we generate a set of candidate entities C m = { c j } ⊂ E using Cross-Wikis (Spitkovsky and Chang, 2012), a dictionary computed from a Google crawl of the web that stores the frequency with which a mention links to a particular entity.”
Collective Entity Disambiguation with Structured Gradient Tree Boosting (2018, methodology): “Following Ganea and Hofmann (2017), the prior is computed by averaging mention prior probabilities built from mention-entity hyperlink statistics from Wikipedia 3 and a large Web corpus (Spitkovsky and Chang, 2012).”
Deep Neural Architectures for Joint Named Entity Recognition and Disambiguation (2019, background): “These candidate entities have prior scores derived from a pre-computed frequency dictionary [18].”
More Accurate Question Answering on Freebase (2015, methodology): “We use a map from phrases (the aliases) to lists of entities (the entities with the respective alias) obtained from the CrossWikis dataset [20].”
Dynamic Knowledge-Base Alignment for Coreference Resolution (2013, background): “lization of a larger set of surface string variations for each entity candidate by using links that appear all over the web (Spitkovsky and Chang, 2012), (3) A combination of these approaches that improves upon a competitive baseline without a knowledge base by 1.”