Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity

By Nina Poerner, Ulli Waltinger, Hinrich Schütze. 2019

21 citations recorded by the source corpus.

We address the task of unsupervised Semantic Textual Similarity (STS) by ensembling diverse pre-trained sentence encoders into sentence meta-embeddings. We apply, extend and evaluate different meta-embedding methods from the word embedding literature at the sentence level, including dimensionality reduction (Yin and Schütze, 2016), generalized Canonical Correlation Analysis (Rastogi et al., 2015) and cross-view auto-encoders (Bollegala and Bao, 2018). Our sentence meta-embeddings set a new unsupervised State of The Art (SoTA) on the STS Benchmark and on the STS12-STS16 datasets, with gains of between 3.7% and 6.4% Pearson’s r over single-source systems.

Subfields: Artificial Intelligence, General Social Sciences, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Social Sciences

In Natural Language Processing

Citations in the corpus

Generating Datasets with Pretrained Language Models (2021, methodology): “Ensem-bling several methods improves results (Pörner and Schütze, 2019; Pörner et al., 2020).”
Uncertainty Measures in Neural Belief Tracking and the Effects on Dialogue Policy Performance (2021, methodology): “We choose to make this expansion, as approaches which utilise a sequence of embeddings outperform approaches based on a single embedding in various natural language processing tasks (Poerner et al., 2020; Choi et al., 2021).”