Multi-Step Regression Learning for Compositional Distributional Semantics

By Edward Grefenstette, Georgiana Dinu, Yao-zhong Zhang, Mehrnoosh Sadrzadeh, Marco Baroni. 2013

145 citations recorded by the source corpus.

We present a model for compositional distributional semantics related to the framework of Coecke et al. (2010), and emulating formal semantics by representing functions as tensors and arguments as vectors. We introduce a new learning method for tensors, generalising the approach of Baroni and Zamparelli (2010). We evaluate it on two benchmark data sets, and find it to outperform existing leading methods. We argue in our analysis that the nature of this learning method also renders it suitable for solving more subtle problems compositional distributional models might face.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Distributed Representations of Sentences and Documents (2014, methodology): “Representing phrases is a recent trend and received much attention (Mitchell & Lapata, 2010; Zanzotto et al., 2010; Yessenalina & Cardie, 2011; Grefenstette et al., 2013; Mikolov et al., 2013c).”
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank (2013, background): “Because they cannot capture the meaning of longer phrases properly, compositionality in semantic vector spaces has recently received a lot of attention (Mitchell and Lapata, 2010; Socher et al., 2010; Zanzotto et al., 2010; Yessenalina and Cardie, 2011; Socher et al., 2012; Grefenstette et al., 2013”
A Neural Network for Factoid Question Answering over Paragraphs (2014, background): “Many computational models of compositionality focus on learning vector spaces (Zanzotto et al., 2010; Erk, 2012; Grefenstette et al., 2013; Yessenalina and Cardie, 2011).”
Grounded Compositional Semantics for Finding and Describing Images with Sentences (2014, background): “Compositionality is an active field of research with many different models and representations being explored (Grefenstette et al., 2013), among many others.”
Category-theoretic quantitative compositional distributional models of natural language semantics (2013, background): “The question is now: can this learning procedure be scaled to deal with the higher order tensors used for other constructs, such as transitive verbs? In [35]1, in collaboration with Baroni and his colleagues, Mehrnoosh Sadrzadeh and I have produced a generalised multi-step linear regression learning”
Measuring Compositionality in Representation Learning (2019, background): “(Future work might explore learned composition functions as in e.g. Grefenstette et al., 2013, for future work.)”
“Not not bad” is not “bad”: A distributional account of negation (2013, methodology): “…2008; Mitchell and Lapata, 2010; Zanzotto et al., 2010), matrix or tensor-based approaches (Baroni and Zampar-elli, 2010; Coecke et al., 2010; Grefenstette et al., 2013; Kartsaklis et al., 2012), or through the use of recursive auto-encoding (Socher et al., 2011; Hermann and Blunsom, 2013)…”
Global Belief Recursive Neural Networks (2014, background): “There has recently been a lot of work extending single word semantic vector spaces [27, 11, 16] to compositional models of bigrams [17, 29] or phrases of arbitrary length [25, 28, 24, 10].”
Frege in Space: A Program of Compositional Distributional Semantics (2014, methodology): “53 Grefenstette et al. (2013) illustrated this for transitive verbs.”
The Role of Syntax in Vector Space Models of Compositional Semantics (2013, methodology): “These two approaches are combined in Grefenstette et al. (2013), producing a tensor-based semantic framework with tensor contraction as composition operation.”
Prior Disambiguation of Word Tensors for Constructing Sentence Vectors (2013, methodology): “One can generalize this procedure to tensors of higher order by proceeding step-wise, as done by Grefenstette et al. (2013).”
Challenges in Sentiment Analysis (2017, background): “…has gained substantial attention in recent years with work from Mitchell and Loapata (2010), Baroni and Zamparelli (2010), Rudolph and Giesbrecht (2010), Yessenalina and Cardie (2011), Grefenstette, Dinu, Zhang, Sadrzadeh, and Baroni (2013), Grefenstette and Sadrzadeh (2011), and Turney (2014).”
An Exploration of Discourse-Based Sentence Spaces for Compositional Distributional Semantics (2015, methodology): “The ﬁrst follows Grefenstette et al. (2013) and Fried et al. (2015) in using intra-sentential word co-occurrenes with SVO triples.”
Factor-based Compositional Embedding Models (2014, background): “Previous work on compositional phrase semantics [14, 15] can be seen as special cases of this type of models for phrases.”