UdL at SemEval-2017 Task 1: Semantic Textual Similarity Estimation of English Sentence Pairs Using Regression Model over Pairwise Features

By Hussein T. Al-Natsheh, Lucie Martinet, Fabrice Muhlenbach, Djamel Abdelkader Zighed. 2017

13 citations recorded by the source corpus.

This paper describes the model UdL we proposed to solve the semantic textual similarity task of SemEval 2017 workshop. The track we participated in was estimating the semantics relatedness of a given set of sentence pairs in English. The best run out of three submitted runs of our model achieved a Pearson correlation score of 0.8004 compared to a hidden human annotation of 250 pairs. We used random forest ensemble learning to map an expandable set of extracted pairwise features into a semantic similarity estimated value bounded between 0 and 5. Most of these features were calculated using word embedding vectors similarity to align Part of Speech (PoS) and Name Entities (NE) tagged tokens of each sentence pair. Among other pairwise features, we experimented a classical tf-idf weighted Bag of Words (BoW) vector model but with character-based range of n-grams instead of words. This sentence vector BoW-based feature gave a relatively high importance value percentage in the feature importances analysis of the ensemble learning.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Daniel Matthew Cer
2. Graham Neubig
3. Iryna Gurevych
4. Eneko Agirre
5. Mona T. Diab
6. John Wieting
7. Kevin Gimpel
8. Jianlin Su

In Natural Language Processing

1. John Wieting
2. Kevin Gimpel
3. Daniel Matthew Cer
4. Kawin Ethayarajh
5. Iryna Gurevych
6. Mona T. Diab
7. Eneko Agirre
8. Nils Reimers

In Evaluation & Benchmarks

1. John Wieting
2. Daniel Matthew Cer
3. Kevin Gimpel
4. Kawin Ethayarajh
5. Iryna Gurevych
6. Jianlin Su
7. Mona T. Diab
8. Eneko Agirre

Citations in the corpus

• SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation (2017, methodology): “cosine over PoS and NER based alignments (Al-Natsheh et al., 2017) HCTI Deep learning model with sent.”
• RoFormer: Enhanced Transformer with Rotary Position Embedding (2021, positive): “We look at several datasets from GLUE, i.e. MRPC Dolan and Brockett [2005], SST-2 Socher et al. [2013], QNLI Rajpurkar et al. [2016], STS-B Al-Natsheh [2017], QQP Chen et al. [2018b] and MNLI Williams et al. [2018].”