Multi-Relational Latent Semantic Analysis

By Kai-Wei Chang, Wen-tau Yih, Christopher Meek. 2013

72 citations recorded by the source corpus.

We present Multi-Relational Latent Semantic Analysis (MRLSA) which generalizes Latent Semantic Analysis (LSA). MRLSA provides an elegant approach to combining multiple relations between words by constructing a 3-way tensor. Similar to LSA, a lowrank approximation of the tensor is derived using a tensor decomposition. Each word in the vocabulary is thus represented by a vector in the latent semantic space and each relation is captured by a latent square matrix. The degree of two words having a specific relation can then be measured through simple linear algebraic operations. We demonstrate that by integrating multiple relations from both homogeneous and heterogeneous information sources, MRLSA achieves stateof-the-art performance on existing benchmark datasets for two relations, antonymy and is-a.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yoav Goldberg
2. Quan Wang
3. Li Guo
4. Mohit Bansal
5. Chris Dyer
6. Wen-tau Yih
7. Zhiyuan Liu
8. Noah A. Smith

In Natural Language Processing

1. Chris Dyer
2. Wen-tau Yih
3. Manaal Faruqui
4. Ting-Wei Liu
5. Kai-Wei Chang
6. Bishan Yang
7. Dan Roth
8. Tao Ge

In Representation & Self-Supervised Learning

1. Chris Dyer
2. Manaal Faruqui
3. Quan Wang
4. Li Guo
5. Kai-Wei Chang
6. Ting-Wei Liu
7. Tao Ge
8. Baobao Chang

Citations in the corpus

• Knowledge Graph Embedding by Translating on Hyperplanes (2014, background): “…scoring functions, the implied transformations vary between simple difference (Bordes et al. 2012), translation (Bordes et al. 2013b), affine (Chang, Yih, and Meek 2013), general linear (Bordes et al. 2011), bilinear (Jenatton et al. 2012; Sutskever, Tenenbaum, and Salakhutdinov 2009), and…”
• Improving Vector Space Word Representations Using Multilingual Correlation (2014, methodology): “Our method can also be seen as an application of multi-view learning (Chang et al., 2013; Collobert and Weston, 2008), where one of the views can be used to capture cross-lingual information.”
• Max-Margin Tensor Neural Network for Chinese Word Segmentation (2014, methodology): “Various tensor factorization (decomposition) methods have been proposed recently for tensorbased dimension reduction (Cohen et al., 2013; Van de Cruys et al., 2013; Chang et al., 2013).”
• SSE: Semantically Smooth Embedding for Knowledge Graphs (2017, background): “This section reviews two lines of embedding strategies: 1) strategies based on latent distance (LD) and 2) strategies based on tensor factorization.”
• Knowledge Graph Embedding: A Survey of Approaches and Applications (2017, methodology): “have also been applied on KGs to model multi-relational data [83], [84], [85].”
• Tensors for Data Mining and Data Fusion (2016, background): “A higher-order extension of LSA can be found in Chang et al. [2013], where the authors integrate multiple relations between words from different information sources.”
• Knowledge Graph and Text Jointly Embedding (2014, background): “A knowledge graph is embedded into a low-dimensional continuous vector space while certain properties of it are preserved (Bordes et al., 2011; Bordes et al., 2013; Socher et al., 2013; Chang et al., 2013; Wang et al., 2014).”
• Knowledge-Driven Event Embedding for Stock Prediction (2016, methodology): “Knowledge Graph Embedding Recently, several methods have been explored to represent and encode knowledge graph (Bordes et al., 2013; Bordes et al., 2014; Chang et al., 2013; Ji et al., 2015; Lin et al., 2015) in distributed vectors.”
• Aligning Knowledge and Text Embeddings by Entity Descriptions (2015, background): “It is capable of reasoning missing facts in a KB and helping facts extraction (Bordes et al., 2011; Bordes et al., 2012; Bordes et al., 2013; Socher et al., 2013; Chang et al., 2013; Wang et al., 2014b; Lin et al., 2015).”
• Typed Tensor Decomposition of Knowledge Bases for Relation Extraction (2014, methodology): “Incorporating such information in other tensor decomposition methods (e.g., (Chang et al., 2013)) may help lexical semantic representations.”
• Semantically Smooth Knowledge Graph Embedding (2015, background): “1) The smoothness assumptions can be imposed to a wide variety of KG embedding models, not only the ones introduced in Section 2, but also those based on matrix / tensor factorization (Nickel et al., 2011; Chang et al., 2013).”