SemEval-2020 Task 4: Commonsense Validation and Explanation

By Cunxiang Wang, Shuailong Liang, Yili Jin, Yilong Wang, Xiao-Dan Zhu, Yue Zhang. 2020

105 citations recorded by the source corpus.

In this paper, we present SemEval-2020 Task 4,CommonsenseValidation andExplanation(ComVE), which includes three subtasks, aiming to evaluate whether a system can distinguish anatural language statement thatmakes senseto humans from one that does not, and provide thereasons. Specifically, in our first subtask, the participating systems are required to choose from twonatural language statements of similar wording the one thatmakes senseand the one does not. Thesecond subtask additionally asks a system to select the key reason from three options why a givenstatement does not make sense. In the third subtask, a participating system needs to generate thereason automatically. 39 teams submitted their valid systems to at least one subtask. For SubtaskA and Subtask B, top-performing teams have achieved results closed to human performance.However, for Subtask C, there is still a considerable gap between system and human performance.The dataset used in our task can be found athttps://github.com/wangcunxiang/SemEval2020-Task4-Commonsense-Validation-and-Explanation.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations (2021, methodology): “Commonsense Validation and Explanation (ComVE) (Wang et al., 2020), as reformulated by Majumder et al. (2022), is the task of jointly identifying which one of two statements contradicts commonsense and explaining why.”
CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning (2020, background): “…(Chen et al., 2019), Hel-laSWAG (Zellers et al., 2019b)), cultural and social understanding (Lin et al., 2018; Sap et al., 2019a,b), visual scene comprehension (Zellers et al., 2019a), and general commonsense question answering (Tal-mor et al., 2019; Huang et al., 2019; Wang et al., 2019a, 2020).”
Can Generative Pre-trained Language Models Serve As Knowledge Bases for Closed-book QA? (2021, methodology): “Wang et al. (2019); Zhou et al. (2020) adopt some discriminative PLMs on commonsense reasoning QA tasks such as ComVE (Wang et al., 2020) and Swag (Zellers et al., 2018) without ﬁnetuning, indicating the PLMs have commonsense knowledge.”
Rationale-Inspired Natural Language Explanations with Commonsense (2021, methodology): “For ComVE, we consider task A from [45].”
Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanations (2021, methodology): “Until now, the SOTA models in terms of task performance for all ﬁve tasks were models that do not offer any explain-ability (Wang et al., 2020; 2021; Lan et al., 2020; Xie et al., 2019; Yu et al., 2020).”
A Comprehensive Survey on Automatic Knowledge Graph Construction (2023, background): “4 Advanced Semantic and Dynamic in Knowledge Graph Construction Tasks Recent research has extended to advanced semantic evaluation tasks, such as detecting equivoque [351] and validating facts with common sense [352] to handle complex lingual phenomena.”
Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning (2021, background): “…that pretrained language models (Liu et al., 2019; Lan et al., 2020) can generalize well on textual commonsense reasoning tasks (Wang et al., 2019), Wang et al. (2020b) ﬁnds that transformer models (Bosselut et al., 2019) may not generalize well on commonsense knowledge graph (Sap et al., 2019)…”
Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts (2022, methodology): “The dataset has collected explanations to counterfactual statements for sense-making from three anno-tators (Wang et al., 2020).”