DLS@CU: Sentence Similarity from Word Alignment and Semantic Vector Composition

By Md Arafat Sultan, Steven Bethard, T. Sumner. 2015

116 citations recorded by the source corpus.

We describe a set of top-performing systems at the SemEval 2015 English Semantic Textual Similarity (STS) task. Given two English sentences, each system outputs the degree of their semantic similarity. Our unsupervised system, which is based on word alignments across the two input sentences, ranked 5th among 73 submitted system runs with a mean correlation of 79.19% with human annotations. We also submitted two runs of a supervised system which uses word alignments and similarities between compositional sentence vectors as its features. Our best supervised run ranked 1st with a mean correlation of 80.15%.

Subfields: Artificial Intelligence, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

CompiLIG at SemEval-2017 Task 1: Cross-Language Plagiarism Detection Methods for Semantic Textual Similarity (2017, methodology): “Last year, among 26 submissions from 10 teams, the method that achieved the best performance (Brychcin and Svoboda, 2016) was a supervised system (SVM regression with RBF kernel) based on word alignment algorithm presented in Sultan et al. (2015).”
ECNU at SemEval-2017 Task 1: Leverage Kernel-based Traditional NLP features and Neural Networks to Build a Universal Model for Multilingual and Cross-lingual Semantic Textual Similarity (2017, methodology): “Word Embedding Features: Each sentence is represented by concatenating min/max/average pooling of vector representations of words.”
UWB at SemEval-2016 Task 1: Semantic Textual Similarity using Lexical, Syntactic, and Semantic Information (2016, background): “• Word alignment: Method presented in (Sultan et al., 2014a; Sultan et al., 2014b; Sultan et al., 2015) has been very successful in last years.”
An Empirical Evaluation of doc2vec with Practical Insights into Document Embedding Generation (2016, methodology): “Unsurprisingly, we do not exceed the overall performance of the supervised benchmark system DLS , although doc2vec outperforms DLS over Comparing doc2vec and word2vec , doc2vec performs better.”
FCICU at SemEval-2017 Task 1: Sense-Based Language Independent Semantic Textual Similarity Approach (2017, methodology): “Many of those methods were very successful and were among the top performing methods during the last years of SemEval 2013-2016 (Han et al., 2013; Han et al., 2015; Hänig et al., 2015; Sultan et al., 2014a; Sultan et al., 2014b; Sultan et al., 2015).”
Bilingual Embeddings and Word Alignments for Translation Quality Estimation (2016, methodology): “According to the evaluation of the semantic textual similarity task in SemEval 2015, the best performing systems in both the English and Spanish subtasks relied mainly on word alignment techniques (Sultan et al., 2015; Hänig et al., 2015).”
Evolution of Semantic Similarity—A Survey (2020, methodology): “Word-Alignment models [90]: Word-Alignment models calculate the semantic similarity of sentences based on their alignment over a larger corpus.”
BIT at SemEval-2017 Task 1: Using Semantic Information Space to Evaluate Semantic Textual Similarity (2017, methodology): “…are appeared for a long time when there wasn’t enough training data; The other kind is supervised systems (B¨ar et al., 2012; ˇSari´c et al., 2012; Sultan et al., 2015; Rychalska et al., 2016; Brychc´ın and Svoboda, 2016) applying machine learning algorithms, including deep learning, after…”
STS-UHH at SemEval-2017 Task 1: Scoring Semantic Textual Similarity Using Supervised and Unsupervised Ensemble (2017, methodology): “Inspired by (Sultan et al., 2015; Brychcı́n and Svoboda, 2016), our unsupervised solution calculates”