GraDA: Graph Generative Data Augmentation for Commonsense Reasoning

By Adyasha Maharana, Mohit Bansal. 2022

109 citations recorded by the source corpus.

Recent advances in commonsense reasoning have been fueled by the availability of large-scale human annotated datasets. Manual annotation of such datasets, many of which are based on existing knowledge bases, is expensive and not scalable. Moreover, it is challenging to build augmentation data for commonsense reasoning because the synthetic questions need to adhere to real-world scenarios. Hence, we present GraDA, a graph-generative data augmentation framework to synthesize factual data samples from knowledge graphs for commonsense reasoning datasets. First, we train a graph-to-text model for conditional generation of questions from graph entities and relations. Then, we train a generator with GAN loss to generate distractors for synthetic questions. Our approach improves performance for SocialIQA, CODAH, HellaSwag and CommonsenseQA, and works well for generative tasks like ProtoQA. We show improvement in robustness to semantic adversaries after training with GraDA and provide human evaluation of the quality of synthetic datasets in terms of factuality and answerability. Our work provides evidence and encourages future research into graph-based generative data augmentation.

Subfields: Artificial Intelligence, Natural Language Processing, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Graph Learning

Citations in the corpus

Self-Instruct: Aligning Language Models with Self-Generated Instructions (2022, background): “A variety of works have proposed using LMs for data generation (Schick and Schütze, 2021; Wang et al., 2021; Liu et al., 2022; Meng et al., 2023) or augmentation (Feng et al., 2021; Yang et al., 2020; Mekala et al., 2022).”
Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor (2022, methodology): “A common approach is to automatically augment existing datasets (Anaby-Tavor et al., 2020; Andreas, 2020; Yang et al., 2020; Kaushik et al., 2020; Lee et al., 2021, inter alia).”
Generating Training Data with Language Models: Towards Zero-Shot Language Understanding (2022, background): “Moreover, different from previous studies [1, 76] that rely on labeled data to fine-tune the generative PLM, we directly use prompts to guide data generation without fine-tuning.”
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants (2021, background): “A long line of prior work has trained generative models for question answering (Du et al., 2017; Du and Cardie, 2018; Zhao et al., 2018; Lewis and Fan, 2019; Alberti et al., 2019; Puri et al., 2020; Yang et al., 2020; Bartolo et al., 2021; Lewis et al., 2021).”
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models (2021, background): “Some works explore automatic data synthesis/expansion by ﬁnetuning LMs on existing labeled data (Anaby-Tavor et al., 2020; Papaniko-laou and Pierleoni, 2020; Kumar et al., 2020; Yang et al., 2020), but are limited by data quality.”
True Few-Shot Learning with Language Models (2021, methodology): “Data augmentation techniques assume there is a viable way to create more data from limited data [70–73].”
Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation (2021, methodology): “We use inﬂuence functions (Cook and Weisberg, 1982; Koh and Liang, 2017) to estimate the effect on the validation loss of including a synthetic example as explored by Yang et al. (2020), but adapted for QA.”
Explanations for CommonsenseQA: New Dataset and Models (2021, background): “…goal behind original CQA task is conﬁned only till answering the questions and hence almost all the submissions (Ma et al., 2019; Khashabi et al., 2020; Zhu et al., 2020; Yang et al., 2020) to the leader-board of the CQA dataset focus just on answering the question and not generating explanations.”
Substructure Substitution: Structured Data Augmentation for NLP (2021, methodology): “…generating new examples by training a conditional text generation model (Bergmanis et al., 2017; Liu et al., 2020a; Ding et al., 2020; Liu et al., 2020b, inter alia ), or applying post-processing on the examples generated by pretrained models (Yang et al., 2020; Wan et al., 2020; Yoo et al., 2020).”
STraTA: Self-Training with Task Augmentation for Better Few-shot Learning (2021, background): “Yang et al. (2020) show that such a generative approach consistently outperforms previous data augmentation methods based on back-translation (Sennrich et al., 2016; Xie et al., 2020a).”
Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning (2022, background): “…the PLMs can be either fine-tuned on the labeled samples to learn label-conditioned generation probability (Kumar et al., 2020; Lee et al., 2021; Yang et al., 2020) or take the labeled data as demonstrations (Wang et al., 2021; Yoo et al., 2021) to generate similar samples pertaining to the…”
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation (2022, methodology): “Recently, generative language models have been used to synthesize examples for various NLP tasks (Kumar et al., 2020; Anaby-Tavor et al., 2020; Puri et al., 2020; Yang et al., 2020).”
A Survey on Data Augmentation for Text Classification (2021, unknown): “Generally, Yang et al. [146] propose a generative method that is suitable for increasing the dataset size for question-answering tasks.”