Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding

By Brian Murphy, Partha P. Talukdar, Tom Michael Mitchell. 2012

193 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Visualizing and Understanding Neural Models in NLP (2015, background): “Sequences of words (e.g., phrases and sentences) are also presented in a more structured way than arrangements of pixels.”
Understanding Neural Networks through Representation Erasure (2016, background): “…et al., 2015), interpreting semantic dimensions by presenting humans with a list of words and asking them to choose outliers (Fyshe et al., 2015; Murphy et al., 2012), linking dimensions with semantic lexicons or word properties (Faruqui et al., 2014; Herbelot and Vecchi, 2015), learning sparse…”
Sparse Overcomplete Word Vector Representations (2015, background): “On the surface, discrete theories seem in-commensurate with the distributed approach, a problem now receiving much attention in computational linguistics (Lewis and Steedman, 2013; Kiela and Clark, 2013; Vecchi et al., 2013; Grefen-stette, 2013; Lewis and Steedman, 2014; Paperno et al., 2014).”
Rotated Word Vector Representations and their Interpretability (2017, background): “Previous research on vector representation of words has proposed improving interpretability while keeping the expressive performance by inducing sparsity in word vector dimensions (Murphy et al., 2012; Fyshe et al., 2014).”
Semantic Structure and Interpretability of Word Embeddings (2017, methodology): “in [22], [23] proposed an online method of learning interpretable word embeddings.”
Simultaneously Uncovering the Patterns of Brain Regions Involved in Different Story Reading Subprocesses (2014, methodology): “…in the sentence it belongs to), semantic features of individual words (derived from word dependency and document co-occurrence frequencies in large online corpora and then compressed using Non-Negative Sparse Embedding [17]), and low level percepts such as the number of letters in the current word.”
Relation Extraction with Matrix Factorization and Universal Schemas (2013, background): “Likewise, matrix factorization in various flavors has received significant attention in the lexical semantics community, from LSA to recent work on non-negative sparse embeddings (Murphy et al., 2012).”
On internal language representations in deep learning: an analysis of machine translation and speech recognition (2018, background): “Using human behavioral experiments may be a reasonable proxy [60, 113, 249].”
A Compositional and Interpretable Semantic Space (2015, methodology): “Though the ﬁnal solution may not be globally optimal, this method is capable of handling large amounts of data and has been shown to produce useful solutions in practice (Mairal et al., 2010; Murphy et al., 2012).”
Linear Algebraic Structure of Word Senses, with Applications to Polysemy (2016, background): “” One line of work tries to capture word senses by augmenting embeddings into more complicated representations than a single vector (e.g., (Murphy et al., 2012; Huang et al., 2012)).”