SemEval-2018 Task 3: Irony Detection in English Tweets

By Cynthia Van Hee, Els Lefever, Veronique Hoste. 2018

370 citations recorded by the source corpus.

This paper presents the first shared task on irony detection: given a tweet, automatic natural language processing systems should determine whether the tweet is ironic (Task A) and which type of irony (if any) is expressed (Task B). The ironic tweets were collected using irony-related hashtags (i.e. #irony, #sarcasm, #not) and were subsequently manually annotated to minimise the amount of noise in the corpus. Prior to distributing the data, hashtags that were used to collect the tweets were removed from the corpus. For both tasks, a training corpus of 3,834 tweets was provided, as well as a test set containing 784 tweets. Our shared tasks received submissions from 43 teams for the binary classification Task A and from 31 teams for the multiclass Task B. The highest classification scores obtained for both subtasks are respectively F1= 0.71 and F1= 0.51 and demonstrate that fine-grained irony classification is much more challenging than binary irony detection.

Subfields: Artificial Intelligence, Natural Language Processing, Sentiment & Opinion Analysis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Luis Espinosa Anke
  2. Francesco Barbieri
  3. José Camacho-Collados
  4. Alexandros Potamianos
  5. Wenxuan Zhang
  6. Lidong Bing
  7. Leonardo Neves
  8. Alexandra Chronopoulou

In Natural Language Processing

  1. Luis Espinosa Anke
  2. Francesco Barbieri
  3. José Camacho-Collados
  4. Alexandros Potamianos
  5. Leonardo Neves
  6. Anh Gia-Tuan Nguyen
  7. Luis Espinosa-Anke
  8. Thanh Vu

In Sentiment & Opinion Analysis

  1. José Camacho-Collados
  2. Anh Gia-Tuan Nguyen
  3. Thanh Vu
  4. Francesco Barbieri
  5. Alexandros Potamianos
  6. Alexandra Chronopoulou
  7. Dat Quoc Nguyen
  8. Christos Baziotis

Citations in the corpus