Lossy Compression for Lossless Prediction
By Yann Dubois, Benjamin Bloem-Reddy, Chris J. Maddison. 2021
77 citations recorded by the source corpus.
Most data is automatically collected and only ever"seen"by algorithms. Yet, data compressors preserve perceptual fidelity rather than just the information needed by algorithms performing downstream tasks. In this paper, we characterize the bit-rate required to ensure high performance on all predictive tasks that are invariant under a set of transformations, such as data augmentations. Based on our theory, we design unsupervised objectives for training neural compressors. Using these objectives, we train a generic image compressor that achieves substantial rate savings (more than $1000\times$ on ImageNet) compared to JPEG on 8 datasets, without decreasing downstream classification performance.
Subfields: Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Chenlin Meng
- Stefano Ermon
- Yutong He
- David B. Lobell
- Luc Van Gool
- Marshall Burke
- Patrick Liu
- Yezhen Cong
Citations in the corpus
- Information-theoretic Online Memory Selection for Continual Learning (2022, neutral): “representations based on which the predictions are made by simple modules such as linear layers (Krizhevsky et al., 2012; Dubois et al., 2021).”
- SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery (2022, background): “In recent years, self-supervised learning techniques have quickly become the norm for pre-training models on large-scale natural image datasets [1, 2, 3, 4, 5, 6, 7, 8], and have demonstrated strong performance on downstream tasks including image classification [3, 4, 9, 10], image segmentation [3, ”