“Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding

By Ben Zhou, Daniel Khashabi, Qiang Ning, Dan Roth. 2019

238 citations recorded by the source corpus.

Understanding time is crucial for understanding events expressed in natural language. Because people rarely say the obvious, it is often necessary to have commonsense knowledge about various temporal aspects of events, such as duration, frequency, and temporal order. However, this important problem has so far received limited attention. This paper systematically studies this temporal commonsense problem. Specifically, we define five classes of temporal commonsense, and use crowdsourcing to develop a new dataset, MCTACO, that serves as a test set for this task. We find that the best current methods used on MCTACO are still far behind human performance, by about 20%, and discuss several directions for improvement. We hope that the new dataset and our study here can foster more future research on this topic.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Time-Aware Language Models as Temporal Knowledge Bases (2021, background): “Additionally, we did not test the ability to model temporal expressions such as “before” or “during”, and we did not investigate temporal commonsense (e.g., Zhou et al. 2019), temporal ordering (e.g., Ning et al. 2020) events (e.g., Zhou et al. 2021).”
Cross-Task Generalization via Natural Language Crowdsourcing Instructions (2021, methodology): “We conduct error analysis on 3 distinct tasks (answer generation from Winogrande; question generation from QASC; incorrect answer generation for MC-TACO event-duration questions).”
GLM-130B: An Open Bilingual Pre-trained Model (2022, methodology): “…et al., 2020), ARC (Clark et al., 2018), and OpenbookQA (Mihaylov et al., 2018) in the MIP training, we select another two widely adopted commonsense reasoning datasets in our evaluation: Commonsense QA (Talmor et al., 2019) and Multiple-choice Temporal Commonsense (MC-TACO, Zhou et al. (2019)).”
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP (2021, background): “…et al. 2019, 2020 liar cls/fact checking Wang 2017 limit other Manotas et al. 2020 math_qa qa/multiple-choice qa Amini et al. 2019 mc_taco qa/binary Zhou et al. 2019 medical_questions_pairs cls/paraphrase McCreery et al. 2020 mocha other/regression Chen et al. 2020a multi_news cg/summarization…”
Evaluating Models’ Local Decision Boundaries via Contrast Sets (2020, methodology): “, 2019) • MC-TACO (Zhou et al., 2019)”
Aligning AI With Shared Human Values (2020, background): “This is different from previous commonsense prediction tasks that assess knowledge of what is (descriptive knowledge) (Zhou et al., 2019; Bisk et al., 2019), but which do not assess knowledge of what should be (normative knowledge).”
ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning (2020, methodology): “, 2020) and MCTACO (Zhou et al., 2019) are recent MRC datasets that attempt to reason about event temporal relations using natural language rather than ERE formalism.”
Unsupervised Commonsense Question Answering with Self-Talk (2020, methodology): “MC-TACO: Multiple Choice Temporal commonsense (Zhou et al., 2019).”
Reframing Instructional Prompts to GPTk’s Language (2021, background): “generating incorrect answers to event transience questions MC-TACO (Zhou et al., 2019) Incorrect Answer Generation (IAG) generating incorrect answers to event duration questions MC-TACO (Zhou et al.”
Evaluating NLP Models via Contrast Sets (2020, methodology): “MC-TACO A dataset of reading comprehension questions about multiple temporal common-sense phenomena (Zhou et al., 2019).”
Continual Learning of Large Language Models: A Comprehensive Survey (2024, unknown): “After discussing general observations on CFT in Section 4.3.1, we will delve into each sub-category in detail.”
Large-scale Multi-modal Pre-trained Models: A Comprehensive Survey (2023, background): “McTaco defines five classes of temporal commonsense [237], while TRACIE evaluates models’ temporal understanding of implicit events [238].”
DAGN: Discourse-Aware Graph Network for Logical Reasoning (2021, background): “DROP (Dua et al., 2019) and MA-TACO (Zhou et al., 2019) need the models to have numerical reasoning.”
Temporal Common Sense Acquisition with Minimal Supervision (2020, methodology): “Recently, MCTACO (Zhou et al., 2019) summarizes five types of TCS and the three temporal dimensions studied here are all in their proposal.”
A Survey of Deep Learning for Mathematical Reasoning (2022, unknown): “McTaco (Zhou et al., 2019) studies temporal commonsense problems, while Fermi (Kalyan et al., 2021) studies Fermi problems whose answers can only be approximately estimated.”
Improving In-Context Few-Shot Learning via Self-Supervised Training (2022, methodology): “CosmosQA (Huang et al., 2019), DROP (Dua et al., 2019), EssentialTerms (Khashabi et al., 2017), MCTACO (Zhou et al., 2019), MultiRC (Khashabi et al., 2018), QASC (Khot et al., 2020), Quoref (Dasigi et al., 2019), ROPES (Lee et al., 2021) and Winogrande (Sakaguchi et al., 2020).”