SemEval-2022 Task 2: Multilingual Idiomaticity Detection and Sentence Embedding
By Harish Tayyar Madabushi, Edward Gow-Smith, Marcos García, Carolina Scarton, Marco Idiart, Aline Villavicencio. 2022
74 citations recorded by the source corpus.
This paper presents the shared task on Multilingual Idiomaticity Detection and Sentence Embedding, which consists of two subtasks: (a) a binary classification task aimed at identifying whether a sentence contains an idiomatic expression, and (b) a task based on semantic text similarity which requires the model to adequately represent potentially idiomatic expressions in context. Each subtask includes different settings regarding the amount of training data. Besides the task description, this paper introduces the datasets in English, Portuguese, and Galician and their annotation procedure, the evaluation metrics, and a summary of the participant systems and their results. The task had close to 100 registered participants organised into twenty five teams making over 650 and 150 submissions in the practice and evaluation phases respectively.
Subfields: Artificial Intelligence, Language and Linguistics, Natural Language Processing, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Luke S. Zettlemoyer
- Mohit Bansal
- Yejin Choi
- Noah A. Smith
- Yonatan Belinkov
- Percy Liang
- Graham Neubig
- Jianfeng Gao
In Language and Linguistics
- Yada Pruksachatkun
- Anna Rogers
- Samson Tan
- Verena Rieser
- Douwe Kiela
- Yonatan Belinkov
- Sheng Shen
- Jekaterina Novikova
In Natural Language Processing
- Mohit Bansal
- Yonatan Belinkov
- Yejin Choi
- Noah A. Smith
- Sebastian Gehrmann
- Graham Neubig
- Samuel R. Bowman
- Percy Liang
In Evaluation & Benchmarks
- Sebastian Gehrmann
- Verena Rieser
- Yonatan Belinkov
- Jekaterina Novikova
- Ehud Reiter
- Miruna Clinciu
- Ondrej Dusek
- Zdeněk Kasner
Citations in the corpus
- BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, methodology): “Semantic textual similarity on STS22 (Madabushi et al., 2022) scored using spearman correlation of cosine similarities”
- IdioLink: Retrieving Meaning Beyond Words Across Idiomatic and Literal Expressions (2026, unknown): “…datasets has been proposed for these tasks, including VNC-Tokens (Cook et al., 2008), IDIX (Sporleder et al., 2010), SemEval benchmarks (Ko-rkontzelos et al., 2013; Tayyar Madabushi et al., 2022), and larger modern resources such as MAG-PIE (Haagsma et al., 2020) and EPIe (Saxena and Paul, 2020).”