Deep Neural Networks and Tabular Data: A Survey

By Vadim Borisov, Tobias Leemann, Kathrin Sessler, Johannes Haug, Martin Pawelczyk, Gjergji Kasneci. IEEE Transactions on Neural Networks and Learning Systems, 2021

1,211 citations recorded by the source corpus.

Heterogeneous tabular data are the most commonly used form of data and are essential for numerous critical and computationally demanding applications. On homogeneous datasets, deep neural networks have repeatedly shown excellent performance and have therefore been widely adopted. However, their adaptation to tabular data for inference or data generation tasks remains highly challenging. To facilitate further progress in the field, this work provides an overview of state-of-the-art deep learning methods for tabular data. We categorize these methods into three groups: data transformations, specialized architectures, and regularization models. For each of these groups, our work offers a comprehensive overview of the main approaches. Moreover, we discuss deep learning approaches for generating tabular data and also provide an overview over strategies for explaining deep models on tabular data. Thus, our first contribution is to address the main research streams and existing methodologies in the mentioned areas while highlighting relevant challenges and open research questions. Our second contribution is to provide an empirical comparison of traditional machine learning methods with 11 deep learning approaches across five popular real-world tabular datasets of different sizes and with different learning objectives. Our results, which we have made publicly available as competitive benchmarks, indicate that algorithms based on gradient-boosted tree ensembles still mostly outperform deep learning models on supervised learning tasks, suggesting that the research progress on competitive deep learning models for tabular data is stagnating. To the best of our knowledge, this is the first in-depth overview of deep learning approaches for tabular data; as such, this work can serve as a valuable starting point to guide researchers and practitioners interested in deep learning with tabular data.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3acff13163f51765bb36147f6107967765509d9b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Why do tree-based models still outperform deep learning on tabular data? (2022, positive): “We include this model because it was the best performing deep model in Borisov et al. [2021], and to investigate the impact of inter-sample attention, which performs well on tabular data according to Kossen et al. [2022].”
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, neutral): “Tabular data [593] in vectors of heterogeneous features is essential for numerous industrial applications, and time series data [594] is predominant in many real-world applications including healthcare and manufacturing.”
LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks (2022, neutral): “We consider standard learning algorithms [45, 46].”
ADBench: Anomaly Detection Benchmark (2022, result): “The great performance of tree-based ensembles (in tabular AD) is consistent with the ﬁndings in literature [20, 58, 170], which may be credited to their capacity to handle imbalanced AD datasets via aggregation.”
TabLLM: Few-shot Classification of Tabular Data with Large Language Models (2022, positive): “We systematically iden-tiﬁed datasets from Kadra et al. (2021), Grinsztajn et al. (2022), and Borisov et al. (2022a).”
Accurate predictions on small data with a tabular foundation model (2025, positive): “Although standard neural networks, which unlike TabPFN do not use ICL, were shown to underperform for small ( 10,000 samples) tabular data 1,14,71 , as a point of reference, we still consider a simple neural network, the MLP. Tabular dataset benchmarks.”
When Do Neural Nets Outperform Boosted Trees on Tabular Data? (2023, methodology): “Across experiments on eleven datasets, they conclude that there is still no universal winner among GBDTs and NNs. Borisov et al. [7] compare classical machine learning methods with eleven deep learning approaches on five tabular datasets, concluding that GBDTs still have the edge.”
Language Models are Realistic Tabular Data Generators (2022, methodology): “Most modern deep-learning approaches for tabular data generation built on generative models transferred from the computer vision domain (Borisov et al., 2021), such as Variational Autoencoders (VAEs, Kingma & Welling, 2013) or Generative Adversarial Networks (GANs, Goodfellow et al.”
TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second (2022, background): “1 We note that 241 previous works have found that DL baselines do not outperform or match the performance of GBDT 242 or AutoML methods for small to medium sized tabular data [5, 16, 42].”
Why do tree-based models still outperform deep learning on typical tabular data? (2022, neutral): “…of the random search. findings do not contradict papers claiming benefits from regularization for tabular data [Shavitt and Segal, 2018, Borisov et al., 2021, Kadra et al., 2021b, Lounici et al., 2021], as adequate regularization and careful optimization may allow neural networks to…”
A Fair Benchmarking of Deep Relational Database Learning Models (2026, unknown): “The tabular learning community has benefited from extensive benchmarking studies [5, 6, 7, 8].”
pTNAS: Progressive Neural Architecture Search for Tabular Data (2024, unknown): “…spatial or sequential structures, tabular data is characterized by heterogeneous features and a lack of structural priors, making it far from trivial to capture complex, non-intuitive feature interactions through simple architectural statistics at initialization (Borisov et al., 2024; Kang, 2025).”