Extracting Commonsense Properties from Embeddings with Limited Human Guidance

By Yiben Yang, L. Birnbaum, Ji-Ping Wang, Doug Downey. 2018

21 citations recorded by the source corpus.

Intelligent systems require common sense, but automatically extracting this knowledge from text can be difficult. We propose and assess methods for extracting one type of commonsense knowledge, object-property comparisons, from pre-trained embeddings. In experiments, we show that our approach exceeds the accuracy of previous work but requires substantially less hand-annotated knowledge. Further, we show that an active learning approach that synthesizes common-sense queries can boost accuracy.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

oLMpics-On What Language Model Pre-training Captures (2019, background): “, have recently drawn attention (Yang et al., 2018b; Welbl et al., 2017; Talmor and Berant, 2018) as a challenging task for contemporary models.”
“Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding (2019, background): “…in recent years and existing NLP works have mainly investigated the acquisition and evaluation of commonsense in the physical world, including but not limited to, size, weight, and strength (Forbes and Choi, 2017), roundness and deliciousness (Yang et al., 2018), and intensity (Cocos et al., 2018).”
Do NLP Models Know Numbers? Probing Numeracy in Embeddings (2019, methodology): “For instance, embeddings may capture the size of objects (Forbes and Choi, 2017), speed of vehicles, and many other “com-monsense” phenomena (Yang et al., 2018).”
G-DAug: Generative Data Augmentation for Commonsense Reasoning (2020, background): “, 2019), capture some common sense expressed implicitly in their pretraining corpus, building upon promising evidence from previous work (Yang et al., 2018; Trinh and Le, 2018; Bosselut et al., 2019; Davison et al., 2019).”
Do Language Embeddings capture Scales? (2020, methodology): “Transfer experiments On the F&C relative comparison task (Table 2), rgr+NumBERT performed best, approaching the performance of using DoQ as an oracle, though short of specialized models for this task (Yang et al., 2018).”
How Large Are Lions? Inducing Distributions over Quantitative Attributes (2019, unknown): “There is a line of work (Forbes and Choi, 2017; Yang et al., 2018) to determine the quantitative relation between two nouns on a specific scale.”
Do Language Embeddings capture Scales? (2020, methodology): “Transfer experiments On the F&C relative comparison task (Table 2), rgr+NumBERT performed best, approaching the performance of using DoQ as an oracle, though short of specialized models for this task (Yang et al., 2018).”
Machine Common Sense Concept Paper (2018, background): “6 Approved for Public Release, Distribution Unlimited These researchers are discovering new techniques for extracting commonsense knowledge from language [42][43][44][45], vision [46][47][48][49], and robotics [50][51][52].”
Temporal Common Sense Acquisition with Minimal Supervision (2020, background): “These works include but are not limited to size, weight, and strength (Bagherinezhad et al., 2016; Forbes and Choi, 2017; Elazar et al., 2019), roundness and deli-ciousness (Yang et al., 2018), and intensity (Cocos et al., 2018).”