Language Models are Realistic Tabular Data Generators

By Vadim Borisov, Kathrin Sessler, Tobias Leemann, Martin Pawelczyk, Gjergji Kasneci. International Conference on Learning Representations, 2022

423 citations recorded by the source corpus.

Tabular data is among the oldest and most ubiquitous forms of data. However, the generation of synthetic samples with the original data's characteristics remains a significant challenge for tabular data. While many generative models from the computer vision domain, such as variational autoencoders or generative adversarial networks, have been adapted for tabular data generation, less research has been directed towards recent transformer-based large language models (LLMs), which are also generative in nature. To this end, we propose GReaT (Generation of Realistic Tabular data), which exploits an auto-regressive generative LLM to sample synthetic and yet highly realistic tabular data. Furthermore, GReaT can model tabular data distributions by conditioning on any subset of features; the remaining features are sampled without additional overhead. We demonstrate the effectiveness of the proposed approach in a series of experiments that quantify the validity and quality of the produced data samples from multiple angles. We find that GReaT maintains state-of-the-art performance across numerous real-world and synthetic data sets with heterogeneous feature types coming in various sizes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/394bd431e522b86581086bcb5cd9be161cf1cdf4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yu Meng
2. Chao Zhang
3. Hunter Lang
4. Stefan Hegselmann
5. Jieyu Zhang
6. Yue Yu
7. Monica Agrawal
8. David A. Sontag

In Computer Vision and Pattern Recognition

1. Jieyu Zhang
2. Chao Zhang
3. Yue Yu
4. David A. Sontag
5. Alexander J. Ratner
6. Ranjay Krishna
7. Yu Meng
8. Monica Agrawal

Citations in the corpus

• Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias (2023, background): “Such applications include generating tabular data [4], relation triplets [8], sentence pairs [46], instruction data [40, 50, 53, 47], etc. .”
• TabLLM: Few-shot Classification of Tabular Data with Large Language Models (2022, positive): “Borisov et al. (2022b) introduced an LLM-agnostic method to generate realistic tabular data and found that it achieved better results than existing approaches.”
• TabDLM: Free-Form Tabular Data Generation via Joint Numerical-Language Diffusion (2026, unknown): “These methods serialize each row into a text sequence and fine-tune an autoregressive language model to capture row-level distributions, as exemplified by GReaT (Borisov et al., 2023) with a GPT-2 (Radford et al., 2019) backbone.”