SemEval 2018 Task 2: Multilingual Emoji Prediction

By Francesco Barbieri, José Camacho-Collados, Francesco Ronzano, Luis Espinosa Anke, Miguel Ballesteros, Valerio Basile, Viviana Patti, Horacio Saggion. 2018

161 citations recorded by the source corpus.

This paper describes the results of the first Shared Task on Multilingual Emoji Prediction, organized as part of SemEval 2018. Given the text of a tweet, the task consists of predicting the most likely emoji to be used along such tweet. Two subtasks were proposed, one for English and one for Spanish, and participants were allowed to submit a system run to one or both subtasks. In total, 49 teams participated to the English subtask and 22 teams submitted a system run to the Spanish subtask. Evaluation was carried out emoji-wise, and the final ranking was based on macro F-Score. Data and further information about this task can be found at https://competitions.codalab.org/competitions/17344.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks, Human-Computer Interaction, Philosophy

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

In Human-Computer Interaction

In Philosophy

Citations in the corpus

ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning (2021, background): “– TweetEval: Emotion Recognition (Mohammad et al., 2018), Emoji Prediction (Barbieri et al., 2018), Irony Detection (Van Hee et al., 2018), Hate Speech Detection (Basile et al., 2019), Offensive Language Identification (Zampieri et al., 2019), Sentiment Analysis (Rosenthal et al., 2017), Stance…”
TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification (2020, methodology): “This task consists in, given a tweet, predicting its most likely emoji, and is based on the Emoji Prediction challenge at Semeval2018 (Barbieri et al., 2018).”
TimeLMs: Diachronic Language Models from Twitter (2022, methodology): “The seven tweet classification tasks in TweetEval are emoji prediction (Barbieri et al., 2018), emotion recognition (Mohammad et al.”
XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond (2021, unknown): “…due to, in addition to its well-known uncurated nature (Derczynski et al., 2013), because of discoursive and platform-speciﬁc factors such as out-of-distribution samples, misspellings, slang, vulgarisms, emoji and multimodality, among others (Barbieri et al., 2018; Camacho-Collados et al., 2020).”
Text‐based emotion detection: Advances, challenges, and opportunities (2020, result): “84 It classified emojis into 20 classes and showed improved results in comparison with baseline FastText method.”
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora (2021, methodology): “For downstream tasks, we use a separate set of 1M tweets from each year to construct multi-label hashtag prediction (Gong and Zhang, 2016) datasets and single-label emoji prediction datasets (Barbieri et al., 2018).”