Wikipedia-based Semantic Interpretation for Natural Language Processing

By Evgeniy Gabrilovich, Shaul Markovitch. Journal of Artificial Intelligence Research, 2009

431 citations recorded by the source corpus.

Adequate representation of natural language semantics requires access to vast amounts of common sense and domain-specific world knowledge. Prior work in the field was based on purely statistical techniques that did not make use of background knowledge, on limited lexicographic knowledge bases such as WordNet, or on huge manual efforts such as the CYC project. Here we propose a novel method, called Explicit Semantic Analysis (ESA), for fine-grained semantic interpretation of unrestricted natural language texts. Our method represents meaning in a high-dimensional space of concepts derived from Wikipedia, the largest encyclopedia in existence. We explicitly represent the meaning of any text in terms of Wikipedia-based concepts. We evaluate the effectiveness of our method on text categorization and on computing the degree of semantic relatedness between fragments of natural language text. Using ESA results in significant improvements over the previous state of the art in both tasks. Importantly, due to the use of natural concepts, the ESA model is easy to explain to human users.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/171581afd71133e09aa19a4fe0d6c04c6bff5890

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Latent entity space: a novel retrieval approach for entity-bearing queries (2015, methodology): “Moreover, existing methods like explicit semantic analysis (Gabrilovich and Markovitch 2009) relies on the statistical inference on high-quality knowledge base like Wikipedia, while the effectiveness of collection based entity proﬁle across multiple collections proves that LES could work well even…”
TAGME: on-the-fly annotation of short text fragments (by wikipedia entities) (2010, methodology): “nsists of extending the classic term-based vector-space model with additional dimensions corresponding to features (concepts) extracted from an external knowledge base, such as DMOZ [4, 6], Wikipedia [7, 1, 9], or even the whole Web (such as the Google’s kernel [15]). Probably the best achievements ”
Representation Learning for Measuring Entity Relatedness with Rich Information (2015, methodology): “Traditional methods mainly focus on counting the occurrences of words appearing in each Wikipedia article to construct a high-dimensional semantic space [Salton et al., 1975; Gabrilovich and Markovitch, 2009].”
A framework for benchmarking entity-annotation systems (2013, background): “As a consequence, we only have a partial picture of the eﬃciency and eﬀectiveness of known annotators which makes it diﬃcult to compare them in a fair and complete way.”
Fast and Accurate Annotation of Short Texts with Wikipedia Pages (2010, methodology): “One approach consists of extending the classic term-based vector-space model with additional dimensions corresponding to features (concepts) extracted from an external knowledge base, such as DMOZ [4, 6], Wikipedia [7, 1, 9], or even the whole Web (such as the Google’s kernel [15]).”
Event Detection and Co-reference with Minimal Supervision (2016, background): “Explicit Semantic Analysis ESA uses Wikipedia as an external knowledge base to generate concepts for a given fragment of text (Gabrilovich and Markovitch, 2009).”
BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network (2012, methodology): “Applications using the knowledge contained in Wikipedia include, among others, text categorization [40], computing semantic similarity of texts [40,107,80], coreference resolution [107], multi-document summarization [82], text generation [120] and simplification [132], and clustering web search quer”
Good debt or bad debt: Detecting semantic orientations in economic texts (2013, background): “…enrichment of lexicon with weights for di ﬀ erent concepts and events, an important direction for future research will be to examine how phrase level models can be merged with content models (Gabrilovich & Markovitch, 2010; Malo, Sinha, Wallenius, & Korhonen, 2011; Malo, Si-itari, & Sinha, 2013)).”
Analogical Inference for Multi-relational Embeddings (2017, methodology): “…knowledge bases, such as Freebase (Bollacker et al., 2008), DBpedia (Auer et al., 2007) and Google’s Knowledge Graph (Singhal, 2012), including question-answering (Ferrucci et al., 2010), information retrieval (Dalton et al., 2014) and natural language processing (Gabrilovich & Markovitch, 2009).”
Recursive Feature Generation for Knowledge-based Learning (2018, neutral): “Two examples of such methods are Explicit Semantic Analysis (ESA) (Gabrilovich & Markovitch, 2009), which generates explicit concepts from Wikipedia based on similarity scores, and Word2Vec (Mikolov et al., 2013), which generates latent concepts based on a large corpus.”
Large-scale learning of word relatedness with constraints (2012, methodology): “Method WS-353 MTURK-287 MTURK-771 CLEAR + ESA 0.826 0.746 0.732 CLEAR + TSA 0.850 0.751 0.742 CLEAR + DS 0.809 0.738 0.729 CLEAR + LDA 0.818 0.749 0.731 ESA+TSA 0.772 0.601 0.604 DS + LDA 0.757 0.694 0.638 Table 5: Performance of pairs of methods combined To further compare CLEAR with TSA, we exa”