VIME: Extending the Success of Self- and Semi-supervised Learning to Tabular Domain

By Jinsung Yoon, Yao Zhang, James Jordon, Mihaela van der Schaar. Neural Information Processing Systems, 2020

346 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/1be6ddac2e262ff915c9fc2eacbf615082240219

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

A Cookbook of Self-Supervised Learning (2023, methodology): “Furthermore, it is common with tabular data to predict mask vectors as a pretext task [Yoon et al., 2020, Iida et al., 2021].”
SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training (2021, unknown): “Yoon et al. [49] propose VIME, which employs MLPs in a technique for pre-training based on denoising.”
When Do Neural Nets Outperform Boosted Trees on Tabular Data? (2023, methodology): “During this time, we train and evaluate the algorithm with at most 30 hyperparameter sets (one default set and 29 random sets, using Optuna [2]).”
Why do tree-based models still outperform deep learning on typical tabular data? (2022, unknown): “…adapt deep learning to tabular data: data encoding techniques to make tabular data better suited for deep learning [Hancock and Khoshgoftaar, 2020, Yoon et al., 2020], "hybrid methods" to benefit from the flexibility of neural networks while keeping the inductive biases of other algorithms like…”
Hopular: Modern Hopfield Networks for Tabular Data (2022, methodology): “Value Imputation and Mask Estimation (VIME) uses self- and semi-supervised learning of deep architectures for tabular data (Yoon et al., 2020).”
TabLLM: Few-shot Classification of Tabular Data with Large Language Models (2022, unknown): “Self-supervised objectives have largely revolved around the prediction of masked cells, the iden-tiﬁcation or correction of corrupted cells, and contrastive losses over augmentations (Bahri et al., 2022; Somepalli et al., 2021; Yoon et al., 2020; Arik and Pﬁster, 2021; Huang et al., 2020).”
Why do tree-based models still outperform deep learning on tabular data? (2022, unknown): “…deep learning work on tabular data: data encoding techniques to make tabular data better suited for deep learning [Hancock and Khoshgoftaar, 2020, Yoon et al., 2020], "hybrid methods" to beneﬁt from the ﬂexibility of NNs while keeping the inductive biases of other algorithms like tree-based…”