From Word Embeddings To Document Distances

By Matt J. Kusner, Yu Sun, Nicholas I. Kolkin, Kilian Q. Weinberger. 2015

2,274 citations recorded by the source corpus.

Subfields: Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, background): “metrics are the word mover distance (Kusner et al., 2015), which relies on non-contextual word embeddings, and BERT-SCORE (Zhang et al.”
DART: Open-Domain Structured Data Record to Text Generation (2020, background): “, 2020a), YiSi-1 (Lo, 2019), WMD (Kusner et al., 2015), WMDO (Chow et al.”
GPTScore: Evaluate as You Desire (2023, background): “MoverScore (Zhao et al., 2019) considers both contextual representation and Word Mover’s Distance (Kusner et al., 2015).”
Measuring Data (2022, background): “Earth Mover’s Distance (Rubner et al., 2000) represents the minimal cost for transforming one distribution of into another of the same domain.”
BERTScore: Evaluating Text Generation with BERT (2019, methodology): “Word Mover’s Distance (WMD; Kusner et al., 2015) is a semantic similarity metric that relies on word embeddings and optimal transport.”
Social Bias Frames: Reasoning about Social and Power Implications of Language (2019, methodology): “We additionally compute word mover’s distance (WMD; Kusner et al., 2015), which uses distributed word representations to measure similarity between the generated and target text.”
Learning protein sequence embeddings using information from structure (2019, background): “Word Mover’s Distance [27], in which documents are compared based on an alignment between their words using pretrained word vectors, and it’s supervised variant [28], in which word vector’s are linearly transformed given document class supervision, are also related.”
Relevance-based Word Embedding (2017, background): “Kusner et al. [16] proposed word mover’s distance (WMD), a function for calculating semantic distance be-tween two documents, which measures the minimum traveling distance from the embedded vectors of individual words in one document to the other one.”
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance (2019, methodology): “Word Mover’s Distance (WMD) (Kusner et al., 2015), a special case of Earth Mover’s Distance (Rubner et al.”
Learning Generative Models with Sinkhorn Divergences (2017, background): “It is however important to realize that for large scale and high dimensional learning applications, empirical considerations [9, 22, 13] suggest that, unlike relevant applications of the same scheme in graphics [33], a relatively strong regularization—a large ε—leads not only to more stable results ”
The Fact Extraction and VERification (FEVER) Shared Task (2018, methodology): “Other teams report scoring based on sentence similarity using Word Mover’s Distance (Kusner et al., 2015) or cosine similarity over smooth inverse frequency weightings (Arora et al., 2017), ELMo embeddings (Peters et al., 2018) and TF-IDF (Salton et al., 1983).”
Evaluation of Text Generation: A Survey (2020, background): “Word mover’s distance (WMD; Kusner et al., 2015) is a discrete version of EMD that calculates the distance between two sequences (e.”
Siamese CBOW: Optimizing Word Embeddings for Sentence Representations (2016, positive): “To comply with results reported in other research (Mikolov et al., 2013b; Kusner et al., 2015) we fix the embedding size to 300 and only consider words appearing 5 times or more in the training corpus.”
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation (2022, methodology): “MoverScore (Zhao et al., 2019) measures semantic distance with Word Mover’s Distance (Kusner et al., 2015) on pooled BERT n-gram embeddings.”
From Show to Tell: A Survey on Deep Learning-Based Image Captioning (2021, background): “For example, the WMD score [161], originally introduced to evaluate document semantic dissimilarity, can also be applied to captioning evaluation by considering generated captions and ground-”
Proposal : Scalable , Active and Flexible Learning on Distributions (2015, positive): “Given a distance ρ, one can naturally use k-nearest neighbor models (Póczos, Xiong, and Schneider 2011; Kusner et al. 2015), or Nadaraya-Watson–type local regression models (J.”