Efficient-VDVAE: Less is more

By Louay Hazami, Rayhane Mama, Ragavan Thurairatnam. 2022

32 citations recorded by the source corpus.

Hierarchical VAEs have emerged in recent years as a reliable option for maximum likelihood estimation. However, instability issues and demanding computational requirements have hindered research progress in the area. We present simple modifications to the Very Deep VAE to make it converge up to $2.6\times$ faster, save up to $20\times$ in memory load and improve stability during training. Despite these changes, our models achieve comparable or better negative log-likelihood performance than current state-of-the-art models on all $7$ commonly used image datasets we evaluated on. We also make an argument against using 5-bit benchmarks as a way to measure hierarchical VAE's performance due to undesirable biases caused by the 5-bit quantization. Additionally, we empirically demonstrate that roughly $3\%$ of the hierarchical VAE's latent space dimensions is sufficient to encode most of the image information, without loss of performance, opening up the doors to efficiently leverage the hierarchical VAEs' latent space in downstream tasks. We release our source code and models at https://github.com/Rayhane-mamah/Efficient-VDVAE .

Subfields: Computer Vision and Pattern Recognition, Geophysics, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Fast Lossless Neural Compression with Integer-Only Discrete Flows (2022, methodology): “The models are trained with PyTorch (Paszke et al., 2019) We use the down-sampled Ima-geNet datasets from https://image-net.org/data/ downsample/Imagenet32_train.zip , following Grci ´ c et al. (2021); Hazami et al. (2022).”
Composing Ensembles of Pre-trained Models via Iterative Consensus (2022, methodology): “In Table A3, we compare our approach with a generative model specifically trained on ImageNet, Efficient-VDVAE (Hazami et al., 2022).”