Domain and Function: A Dual-Space Model of Semantic Relations and Compositions

By Peter D. Turney. 2012

197 citations recorded by the source corpus.

Given appropriate representations of the semantic relations between carpenter and wood and between mason and stone (for example, vectors in a vector space model), a suitable algorithm should be able to recognize that these relations are highly similar (carpenter is to wood as mason is to stone; the relations are analogous). Likewise, with representations of dog, house, and kennel, an algorithm should be able to recognize that the semantic composition of dog and house, dog house, is highly similar to kennel (dog house and kennel are synonymous). It seems that these two tasks, recognizing relations and compositions, are closely connected. However, up to now, the best models for relations are significantly different from the best models for compositions. In this paper, we introduce a dual-space model that unifies these two tasks. This model matches the performance of the best previous models for relations and compositions. The dual-space model consists of a space for measuring domain similarity and a space for measuring function similarity. Carpenter and wood share the same domain, the domain of carpentry. Mason and stone share the same domain, the domain of masonry. Carpenter and mason share the same function, the function of artisans. Wood and stone share the same function, the function of materials. In the composition dog house, kennel has some domain overlap with both dog and house (the domains of pets and buildings). The function of kennel is similar to the function of house (the function of shelters). By combining domain and function similarities in various ways, we can model relations, compositions, and other aspects of semantics.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

Fair Is Better than Sensational: Man Is to Doctor as Woman Is to Doctor (2019, background): “3 In such nonfactual, more creative analogies, various terms could be used for completion depending on the implied underlying relation (Turney 2012), which could be unclear or unspeciﬁed in the query.”
SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation (2014, background): “…literature refers to the conceptual relationship between these concepts as association , although it has been given a range of names including relatedness (Budanit-sky and Hirst, 2006; Agirre et al., 2009), topical similarity (Hatzivassiloglou et al., 2001) and domain similarity (Turney, 2012).”
Improving Distributional Similarity with Lessons Learned from Word Embeddings (2015, neutral): “…observation was made by Caron (2001), who suggested adding a parameter p to control the eigenvalue matrix Σ : Later studies show that weighting the eigenvalue matrix Σ d with the exponent p can have a significant effect on performance, and should be tuned (Bullinaria and Levy, 2012; Turney, 2012).”
Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors (2014, background): “…line of DSM research concerns “context engineering”: There has been for example much work on how to encode syntactic information into context features (Padó and Lapata, 2007), and more recent studies construct and combine feature spaces expressing topical vs. functional information (Turney, 2012).”
Dependency-Based Word Embeddings (2014, methodology): “This observation holds for Turing3 and many other nouns as well; BOW find words that associate with w, while DEPS find words that behave like w. Turney (2012) described this distinction as domain similarity versus functional similarity.”
Linguistic Regularities in Sparse and Explicit Word Representations (2014, background): “Turney (2012) assumes a similar setting but with two types of word similarities, and combines them with products and ratios (similar to 3C OS M UL ) to recover a variety of semantic relations, including analogies.”
Multilingual Models for Compositional Distributed Semantics (2014, background): “, 2011; Hermann and Blunsom, 2013), relational similarity (Turney, 2012) or dialogue analy-”
Combining Heterogeneous Models for Measuring Relational Similarity (2013, background): “, 2010), the issue of how to represent and compare pairs of words in a vector space remains unclear (Turney, 2012).”
GraRep: Learning Graph Representations with Global Structural Information (2015, methodology): “We can approximate the original matrix This way, we can factorize our matrix X k as: The resulting W k gives representations of current vertices as its column vectors, and C k gives the representations of context vertices as its column vectors [6, 5, 30].”
Frege in Space: A Program of Compositional Distributional Semantics (2014, unknown): “Vecchi et al. (2011) and Boleda et al.”
Word Embeddings and Gender Stereotypes in Swedish and English (2019, background): “Turney [43] suggests using two adjoined embeddings, one that captures function similarity and one that captures domain similarity.”
How to Train good Word Embeddings for Biomedical NLP (2016, unknown): “tion to primarily capture word function (Turney, 2012).”
A Compositional and Interpretable Semantic Space (2015, background): “VSMs have been shown to closely match human judgements of semantics (for an overview see Sahlgren (2006), Chapter 5), and can be used to study semantic composition (Mitchell and Lapata, 2010; Baroni and Zamparelli, 2010; Socher et al., 2012; Turney, 2012).”
Unsupervised Post-processing of Word Vectors via Conceptor Negation (2018, methodology): “decomposition methods such as singular value decomposition (SVD) and principal component analysis (PCA) are usually used in this line of research (Caron 2001; Bullinaria and Levy 2012; Turney 2012; Levy and Goldberg 2014; Levy, Goldberg, and Dagan 2015; Mu and Viswanath 2018).”
Uncovering Divergent Linguistic Information in Word Embeddings with Lessons for Intrinsic and Extrinsic Evaluation (2018, methodology): “The proposed method is also connected to a similar parameter found in traditional count-based distributional models as introduced by Caron (2001) and further analyzed by Bullinaria and Levy (2012) and Turney (2012).”