Prior Disambiguation of Word Tensors for Constructing Sentence Vectors
By Dimitri Kartsaklis, Mehrnoosh Sadrzadeh. 2013
90 citations recorded by the source corpus.
Recent work has shown that compositionaldistributional models using element-wise operations on contextual word vectors benefit from the introduction of a prior disambiguation step. The purpose of this paper is to generalise these ideas to tensor-based models, where relational words such as verbs and adjectives are represented by linear maps (higher order tensors) acting on a number of arguments (vectors). We propose disambiguation algorithms for a number of tensor-based models, which we then test on a variety of tasks. The results show that disambiguation can provide better compositional representation even for the case of tensor-based models. Furthermore, we confirm previous findings regarding the positive effect of disambiguation on vector mixture models, and we compare the effectiveness of the two approaches.
Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoav Goldberg
- Mohit Bansal
- Phil Blunsom
- Kyunghyun Cho
- Graham Neubig
- Jianfeng Gao
- Noah A. Smith
- Edward Grefenstette
In Natural Language Processing
- Phil Blunsom
- Dimitri Kartsaklis
- Yoav Goldberg
- Edward Grefenstette
- Mohit Bansal
- Kyunghyun Cho
- Jordan L. Boyd-Graber
- Mohit Iyyer
In Representation & Self-Supervised Learning
- Dimitri Kartsaklis
- Phil Blunsom
- Edward Grefenstette
- Mehrnoosh Sadrzadeh
- Matthew Purver
- Dmitrijs Milajevs
- Nal Kalchbrenner
- Jordan L. Boyd-Graber
Citations in the corpus
- A Convolutional Neural Network for Modelling Sentences (2014, background): “, 2010) or to particular word types (Baroni and Zamparelli, 2010; Coecke et al., 2010; Grefenstette and Sadrzadeh, 2011; Kartsaklis and Sadrzadeh, 2013; Grefenstette, 2013).”
- Deep Unordered Composition Rivals Syntactic Methods for Text Classification (2015, background): “Later, their work was extended to take into account the syntactic relation between words (Erk and Padó, 2008; Baroni and Zamparelli, 2010; Kartsaklis and Sadrzadeh, 2013) and grammars (Coecke et al.”
- Evaluating Neural Word Representations in Tensor-Based Compositional Settings (2014, methodology): “The textual content of this dataset is the same as that of (Kartsaklis and Sadrzadeh, 2013), the difference is that th e dataset of (Kartsaklis and Sadrzadeh, 2014) has updated human judgements whereas the previous dataset used the original annotations of the intransitive dataset of (Mitchell a nd L”