TabLLM: Few-shot Classification of Tabular Data with Large Language Models

By Stefan Hegselmann, Alejandro Buendia, Hunter Lang, Monica Agrawal, Xiaoyi Jiang, David A. Sontag. International Conference on Artificial Intelligence and Statistics, 2022

405 citations recorded by the source corpus.

We study the application of large language models to zero-shot and few-shot classification of tabular data. We prompt the large language model with a serialization of the tabular data to a natural-language string, together with a short description of the classification problem. In the few-shot setting, we fine-tune the large language model using some labeled examples. We evaluate several serialization methods including templates, table-to-text models, and large language models. Despite its simplicity, we find that this technique outperforms prior deep-learning-based tabular classification methods on several benchmark datasets. In most cases, even zero-shot classification obtains non-trivial performance, illustrating the method's ability to exploit prior knowledge encoded in large language models. Unlike many deep learning methods for tabular datasets, this approach is also competitive with strong traditional baselines like gradient-boosted trees, especially in the very-few-shot setting.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/9dcee248452d84b6bf26911ba6726ae5ce1a46f3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Fei Wu
2. Jiwei Li
3. Guoyin Wang
4. Xiaoya Li
5. Sujay Khandagale
6. Ben Feuer
7. Micah Goldblum
8. Ganesh Ramakrishnan

Citations in the corpus

• When Do Neural Nets Outperform Boosted Trees on Tabular Data? (2023, background): “Finally, our open-source collection of extensive datasets and metafeatures can make it easier for researchers to design new meta-learned [32] or pre-trained models for tabular data [31, 41, 65].”
• GPT-NER: Named Entity Recognition via Large Language Models (2023, background): “, 2022) have obtained significant performance boosts on a variety of natural language processing tasks (Hegselmann et al., 2022; Vilar et al., 2022; Perez et al., 2021; Pietrzak et al., 2021; Wei et al., 2021).”
• Language Models are Few-shot Learners for Prognostic Prediction (2023, background): “, which has been proven to generate competitive results compared to other LLMs prompting-based regeneration meth-ods by Hegselmann et al. (2022).”
• Griffin: Towards a Graph-Centric Relational Database Foundation Model (2025, unknown): “Models such as TaBERT (Yin et al., 2020) and TabLLM (Hegselmann et al., 2023) integrate text and tabular data to enhance table structure understanding, while TransTab (Wang & Sun, 2022) and XTab (Zhu et al., 2023) explore transfer learning across tables with varying column structures.”