StarSpace: Embed All The Things!

By Ledell Yu Wu, Adam Fisch, Sumit Chopra, Keith Adams, Antoine Bordes, Jason Aaron Edward Weston. 2017

257 citations recorded by the source corpus.

We present StarSpace, a general-purpose neural embedding model that can solve a wide variety of problems: labeling tasks such as text classification,ranking tasks such as information retrieval/web search,collaborative filtering-based or content-based recommendation,embedding of multi-relational graphs, and learning word, sentence or document level embeddings.In each case the model works by embedding those entities comprised of discrete features and comparing them against each other -- learning similarities dependent on the task.Empirical results on a number of tasks show that StarSpace is highly competitive with existing methods, whilst also being generally applicable to new cases where those methods are not.

Subfields: Artificial Intelligence, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

Citations in the corpus

Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring (2019, methodology): “8 (Wu et al., 2018) 56.”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “Nevertheless, the intuitions of Wu et al. [2020b] in recognizing the need to aggregate passage representations does appear to be valid, and in Section 3.”
Personalizing Dialogue Agents: I have a dog, do you have pets too? (2018, methodology): “We use the same representation and loss as in the Starspace model, so without the proﬁle, the two models are identical.”
Fusion of Detected Objects in Text for Visual Question Answering (2019, background): “Dual Encoders, discussed for example by Wu et al. (2018) and Gillick et al. (2018), are models that embed objects of potentially different types into a common representation space where a similarity function can be expressed e.g. as a dot product or a cosine similarity.”
Learning to Speak and Act in a Fantasy Text Adventure Game (2019, background): “Yang et al. (2017) and Bordes et al. (2010) proposed small world setups for instruction following or labeling, but these are much more restricted than the large multi-player text adventure game environment with rich dialogue that we propose here.”
End-to-End Retrieval in Continuous Space (2018, methodology): “The StarSpace model (Wu et al., 2017) extends the idea of learned embeddings to more data types.”
Dialogue Transformers (2019, methodology): “Similarity All dialogue contexts and system actions are embedded into a single semantic vector space and we use the dot-product loss [20, 2, 14] to maximize the similarity with the target label and minimize similarities with sampled incorrect ones.”
Tartan: A retrieval-based socialbot powered by a dynamic finite-state machine architecture (2018, methodology): “We picked question-answer pairs as the training dataset and used StarSpace embeddings [Wu et al., 2017] to encode them.”
PyTorch-BigGraph: A Large-scale Graph Embedding System (2019, background): “842 StarSpace (Wu et al., 2018) - 0.”
Generating Interactive Worlds with Text (2019, methodology): “• Starspace: The Starspace (Wu et al. 2018) model learns a bag-of-words embedding for the location information (e.”
Unsupervised Scalable Representation Learning for Multivariate Time Series (2019, methodology): “These results also indicate that our method consistently outperforms both unsupervised methods TimeNet and RWS (on, respectively, 13 and 9 out of 13 and 12 UCR datasets), showing its performance.”
Context Mover's Distance & Barycenters: Optimal transport of contexts for building representations (2018, background): “, 2015), nodes in a graph (Grover & Leskovec, 2016), sequence data, or other entities (Wu et al., 2017).”
Accelerating Large-Scale Inference with Anisotropic Vector Quantization (2019, methodology): “…we learn embedding functions to map items to be retrieved in a common vector space, where the items can be words, images, users, audio, products, web pages, graph nodes, or anything of interest (Cremonesi et al., 2010; Weston et al., 2010; Guo et al., 2016a; Gillick et al., 2019; Wu et al., 2017).”
Recommendation as a Communication Game: Self-Supervised Bot-Play for Goal-oriented Dialogue (2019, methodology): “To compute such a metric, we train an embedding-driven recommendation model (Wu et al., 2018).”