Associative Compression Networks for Representation Learning
By Alex Graves, Jacob Menick, Aäron van den Oord. 2018
22 citations recorded by the source corpus.
This paper introduces Associative Compression Networks (ACNs), a new framework for variational autoencoding with neural networks. The system differs from existing variational autoencoders (VAEs) in that the prior distribution used to model each code is conditioned on a similar code from the dataset. In compression terms this equates to sequentially transmitting the dataset using an ordering determined by proximity in latent space. Since the prior need only account for local, rather than global variations in the latent space, the coding cost is greatly reduced, leading to rich, informative codes. Crucially, the codes remain informative when powerful, autoregressive decoders are used, which we argue is fundamentally difficult with normal VAEs. Experimental results on MNIST, CIFAR-10, ImageNet and CelebA show that ACNs discover high-level latent features such as object class, writing style, pose and facial expression, which can be used to cluster and classify the data, as well as to generate diverse and convincing samples. We conclude that ACNs are a promising new direction for representation learning: one that steps away from IID modelling, and towards learning a structured description of the dataset as a whole.
Subfields: Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Trevor Darrell
- Dacheng Tao
- Ming-Hsuan Yang
- Andrew Zisserman
- Jitendra Malik
- Xiaogang Wang
- Andrea Vedaldi
Citations in the corpus
- Preventing Posterior Collapse with delta-VAEs (2019, methodology): “The Associative Compression Networks (ACN) (Graves & Menick, 2014) 2017), we use a single channel network to output the components of discretised mixture of logistics distributions for each channel, and linear dependencies between the RGB colour channels.”
- Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis (2018, background): “Chen et al. [43], Husz´ar [44], and Graves et al. [45] provide lucid discussions of this problem.”