Unsupervised Deep Structured Semantic Models for Commonsense Reasoning
By Shuohang Wang, Sheng Zhang, Yelong Shen, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Jing Jiang. 2019
15 citations recorded by the source corpus.
Commonsense reasoning is fundamental to natural language understanding. While traditional methods rely heavily on human-crafted features and knowledge bases, we explore learning commonsense knowledge from a large amount of raw text via unsupervised learning. We propose two neural network models based on the Deep Structured Semantic Models (DSSM) framework to tackle two classic commonsense reasoning tasks, Winograd Schema challenges (WSC) and Pronoun Disambiguation (PDP). Evaluation shows that the proposed models effectively capture contextual information in the sentence and co-reference information between pronouns and nouns, and achieve significant improvement over previous state-of-the-art approaches.
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Weizhu Chen
- Jianfeng Gao
- Xiaodong Liu
- Pengcheng He
- Samuel R. Bowman
- Mohit Bansal
- Zhe Gan
- Luke S. Zettlemoyer
In Natural Language Processing
- Jianfeng Gao
- Xiaodong Liu
- Weizhu Chen
- Pengcheng He
- Caiming Xiong
- Richard Socher
- Samuel R. Bowman
- Ronghang Hu
Citations in the corpus
- A Hybrid Neural Network Model for Commonsense Reasoning (2019, methodology): “Our results show that HNN, combining the strengths of both DSSM and BERT WSCR , has consistently achieved new state-of-the-art results on all three tasks.”
- Contrastive Self-Supervised Learning for Commonsense Reasoning (2020, unknown): “Similarity (USSM) 55.0 % Transformer LM (Vaswani et al., 2017) 58.3 % BERT LM (Trinh and Le, 2018) 60.0 % MAS (Klein and Nabi, 2019) 68.3 % DSSM (Wang et al., 2019) 75.0 % CSS (Proposed Method) 90.0 % WSC-273 (sup.)”