Text: now in 2D! A framework for lexical expansion with contextual similarity

By Chris Biemann, Martin Riedl. 2013

149 citations recorded by the source corpus.

A new metaphor of two-dimensional text for data-driven semantic modeling of natural language is proposed, which provides an entirely new angle on the representation of text: not only syntagmatic relations are annotated in the text, but also paradigmatic relations are made explicit by generating lexical expansions. We operationalize distributional similarity in a general framework for large corpora, and describe a new method to generate similar terms in context. Our evaluation shows that distributional similarity is able to produce highquality lexical resources in an unsupervised and knowledge-free way, and that our highly scalable similarity measure yields better scores in a WordNet-based evaluation than previous measures for very large corpora. Evaluating on a lexical substitution task, we find that our contextualization method improves over a non-contextualized baseline across all parts of speech, and we show how the metaphor can be applied successfully to part-of-speech tagging. A number of ways to extend and improve the contextualization method within our framework are discussed. As opposed to comparable approaches, our framework defines a model of lexical expansions in context that can generate the expansions as opposed to ranking a given list, and thus does not require existing lexical-semantic resources.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

What do we need to build explainable AI systems for the medical domain? (2017, background): “More promising, in our opinion, is the use of hybrid distributional models that combine sparse graph-based representations (Biemann and Riedl, 2013) with dense vector representations Mikolov et al. (2013b) and link them to lexical resources and knowledge bases (Faralli et al., 2016).”
Unsupervised Does Not Mean Uninterpretable: The Case for Word Sense Induction and Disambiguation (2017, methodology): “An open source implementation of the method featuring a live demo of several pre-trained models is available online.”
STS-UHH at SemEval-2017 Task 1: Scoring Semantic Textual Similarity Using Supervised and Unsupervised Ensemble (2017, methodology): “II Distributional Thesaurus (DTs) Expansion Feature: Each non-stopword is expanded to its most similar top 10 words using the API for the Distributional Thesaurus (DTs) by Biemann and Riedl (2013).”
Modeling Word Meaning in Context with Substitute Vectors (2015, methodology): “To the best of our knowledge, Biemann and Riedl (2013) is the only prior work that attempted to perform the original SemEval 2007 task on the LS07 dataset, learning only from corpus data like we do.”
Linked Disambiguated Distributional Semantic Networks (2016, methodology): “1) Learning a JoBimText model: initially, we automatically create a sense inventory from a large text collection using the pipeline of the JoBimText project [2, 22]1.”
Human and Machine Judgements for Russian Semantic Relatedness (2016, methodology): “Biemann and Riedl [27] follow a similar approach based on the English WordNet to assess quality of their distributional semantics framework.”
Making Sense of Word Embeddings (2016, methodology): “This number is motivated by prior studies (Biemann and Riedl, 2013; Panchenko, 2013): as observed, only few words have more strongly semantically re-lated words.”
Do Supervised Distributional Methods Really Learn Lexical Inference Relations? (2015, methodology): “In addition to our setup, these results were also reproduced in preliminary exper-971 iments by applying the JoBimText framework 4 for scalable distributional thesauri (Biemann and Riedl, 2013) using Google’s syntactic N-grams (Goldberg and Orwant, 2013) as a corpus.”
A Simple Word Embedding Model for Lexical Substitution (2015, background): “Finally, our results are on par with the results reported by Biemann and Riedl (2013).”
An automatic approach to identify word sense changes in text media across timescales (2015, background): “The LMI measure was shown to yield the best results amongst several measures for feature ranking in this approach in Biemann and Riedl (2013).”