Retrieval Enhanced Model for Commonsense Generation

By Han Wang, Yang Liu, Chenguang Zhu, Linjun Shou, Ming Gong, Yichong Xu, Michael Zeng. 2021

36 citations recorded by the source corpus.

Commonsense generation is a challenging task of generating a plausible sentence describing an everyday scenario using provided concepts. Its requirement of reasoning over commonsense knowledge and compositional generalization ability even puzzles strong pre-trained language generation models. We propose a novel framework using retrieval methods to enhance both the pre-training and fine-tuning for commonsense generation. We retrieve prototype sentence candidates by concept matching and use them as auxiliary input. For fine-tuning, we further boost its performance with a trainable sentence retriever. We demonstrate experimentally on the large-scale CommonGen benchmark that our approach achieves new state-of-the-art results.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Retrieval-Augmented Generation, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chenguang Zhu
2. Meng Jiang
3. Xiang Ren
4. Bill Yuchen Lin
5. Yichong Xu
6. Wenhao Yu
7. Mohit Bansal
8. Yejin Choi

In Computer Vision and Pattern Recognition

1. Chenguang Zhu
2. Bill Yuchen Lin
3. Xiang Ren
4. Yichong Xu
5. Meng Jiang
6. Wenhao Yu
7. Yejin Choi
8. Hannaneh Hajishirzi

In Retrieval-Augmented Generation

1. Xiang Ren
2. Meng Jiang
3. Yichong Xu
4. Bill Yuchen Lin
5. Chenguang Zhu

In Natural Language Generation

1. Xiang Ren
2. Bill Yuchen Lin

Citations in the corpus

• Knowledge-Augmented Methods for Natural Language Processing (2023, methodology): “…2019; Lv et al., 2020; Yu et al., 2022b); • Knowledge-augmented NLG: (Zhou et al., 2018; Zhang et al., 2020a; Ji et al., 2020b; Lewis et al., 2020; Wang et al., 2021); • Commonsense Knowledge and Reasoning for NLP: (Lin et al., 2019; Ma et al., 2019; Fan et al., 2020; Liu et al., 2021b; Wang…”
• A Survey of Knowledge-enhanced Text Generation (2020, unknown): “Commonsense reasoning [76] CommonGen 67,389 4,018 6,042 ✓§ GEM [32, 80, 126] [7] αNLG-ART 50,481 7,252 2,976 ✓¶ GENIE [7, 57] [124] ComVE 25,596 1,428 2,976 ✓∥ SemEval [56, 57] Summarization [109] CNN/DM 287,226 13,368 11,490 ✓ ∗∗ GLGE [33, 41, 163] [109] Gigaword 3.”