Temporal Common Sense Acquisition with Minimal Supervision

By Ben Zhou, Qiang Ning, Daniel Khashabi, Dan Roth. 2020

96 citations recorded by the source corpus.

Temporal common sense (e.g., duration and frequency of events) is crucial for understanding natural language. However, its acquisition is challenging, partly because such information is often not expressed explicitly in text, and human annotation on such concepts is costly. This work proposes a novel sequence modeling approach that exploits explicit and implicit mentions of temporal common sense, extracted from a large corpus, to build TacoLM, a temporal common sense language model. Our method is shown to give quality predictions of various dimensions of temporal common sense (on UDST and a newly collected dataset from RealNews). It also produces representations of events for relevant tasks such as duration comparison, parent-child relations, event coreference and temporal QA (on TimeBank, HiEVE and MCTACO) that are better than using the standard BERT. Thus, it will be an important component of temporal NLP.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Symbolic Knowledge Distillation: from General Language Models to Commonsense Models (2021, background): “Most prior art for automatic knowledge graph construction extracts knowledge from raw text (Bhakthavatsalam et al., 2020; Zhang et al., 2020a; Zhou et al., 2020; Zhang et al., 2020b; Li et al., 2020).”
ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning (2020, methodology): “Recent approaches leveraging large pre-trained language models (PTLMs) achieved state-of-the-art results on a range of event temporal reasoning tasks (Ning et al., 2020; Pereira et al., 2020; Wang et al., 2020; Zhou et al., 2020c; Han et al., 2019b).”
TransOMCS: From Linguistic Graphs to Commonsense Knowledge (2020, background): “, 2019] and [Zhou et al., 2020], and physical attributes of objects [Elazar et al.”
Temporal Reasoning on Implicit Events from Distant Supervision (2020, methodology): “As a task that requires considerable commonsense knowledge, we follow Zhou et al. (2020) in minimizing the size of the training set, therefore making T RACIE mainly an evaluation set.”
Joint Constrained Learning for Event-Event Relation Extraction (2020, methodology): “This is shown to be a comprehensive way to model embedding interactions (Zhou et al., 2020b).”
Do Language Embeddings capture Scales? (2020, background): “, 2020) (”How to separate egg whites from yolks?”), temporal reasoning (Zhou et al., 2020) (”How long does a basketball game take?”), and numerical common sense (Lin et al.”
TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions (2020, background): “While how long and how often usually require temporal common sense knowledge (Vempala et al., 2018; Zhou et al., 2019, 2020), the problem of when often boils down to extracting the temporal relations.”
SituatedQA: Incorporating Extra-Linguistic Contexts into QA (2021, background): “, ordering events or finding temporal relationship between two events in the same document (Pustejovsky et al., 2003; Bethard et al., 2007; Cassidy et al., 2014; Llorens et al., 2015; Zhou et al., 2020).”
Do Language Embeddings capture Scales? (2020, background): “, 2020) (”How to separate egg whites from yolks?”), temporal reasoning (Zhou et al., 2020) (”How long does a basketball game take?”), and numerical common sense (Lin et al.”