Rate Distortion Theory

By Thomas M. Cover, Joy A. Thomas. 2001

522 citations recorded by the source corpus.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/ad0f1314b0f17fb793d25e0adb97ec397c81e05d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

Citations in the corpus

Sparse Coding via Thresholding and Local Competition in Neural Circuits (2008, methodology): “Viewing the sparse approximation problem through the lens of rate-distortion theory (Berger, 1971), the most powerful algorithm produces the lowest reconstruction MSE for a given sparsity.”
Unified Multivariate Gaussian Mixture for Efficient Neural Image Compression (2022, background): “It adopts neural networks as nonlinear transforms to extract binaries from images and restore them, while essential research problem is to handle the trade-off between rate and distortion [7].”
3D High-Efficiency Video Coding for Multi-View Video and Depth Data (2013, unknown): “This is equivalent to minimizing the Lagrangian cost function J = D + λ·R [3], which weights the distortion D that is obtained by coding a block in a particular mode or with a particular set of parameters with the number of bits R that is required for transmitting all data of that mode or parameter.”
Bounded complexity, mean equicontinuity and discrete spectrum (2018, methodology): “Recently, in the investigation of the Sarnak’s conjecture, Huang, Wang and Ye [18] introduced the measure complexity of an invariant measureμ, similar to the one introduced by Katok [21], by using the mean metric instead of the Bowen metric (for discussion and results related to mean metric, see als”
A Unifying Review of Deep and Shallow Anomaly Detection (2020, background): “Similarly, information-theoretic perspectives of VAEs have been formulated showing that these models can be viewed as making a rate-distortion tradeoff [560] when balancing the latent compression (negative rate) and reconstruction accuracy (distortion) [561], [562].”
Multidimensional Waveform Encoding: A New Digital Beamforming Technique for Synthetic Aperture Radar Remote Sensing (2008, background): “Optimum data compression algorithms and architectures can be derived from information theory by employing a rate distortion analysis [39], which has to take into account the desired performance of the final image product.”
End-to-End Learnt Image Compression via Non-Local Attention Optimization and Improved Context Modeling (2019, background): “This is also referred to as the source coding [5].”
Theoretical foundations of transform coding (2001, background): “This accuracy measure is conventional and usually leads to the easiest mathematical results, though the theory of source coding has been developed with quite general measures [1].”
NASH: Toward End-to-End Neural Architecture for Generative Semantic Hashing (2018, methodology): “Motivated by the rate-distortion theory (Berger, 1971; Theis et al., 2017), we propose to inject data-dependent noise into the latent codes during the decoding stage, which adaptively accounts for the tradeoff between minimizing rate (number of bits used, or effective code length) and distortion (re”
The Necessity and Sufficiency of Anytime Capacity for Stabilization of a Linear System Over a Noisy Communication Link—Part I: Scalar Systems (2006, background): “Shannon classical capacityC can also be calculated in the case of memoryless channels by solving an optimization problem: C = sup P(A) I(A; B) where the maximization is over the input probability distribution and I(A; B) represents the mutual information through the channel [1].”
Sharp asymptotics of the functional quantization problem for Gaussian processes (2004, background): “…entropy (mutual information) if Q is absolutely continuous with respect to the product of the marginals P ⊗ Q 2 and equals to ∞ otherwise. coding Berger The simple converse part of the source theorem [cf. (1971), Theorem 3.2.2, and Graf and Luschgy (2000), page 163] says that the minimal number…”
Nonlinear Transform Source-Channel Coding for Semantic Communications (2021, background): “Since the nonlinear transform is not strictly invertible, also the quantization step introduces error, the optimization of NTC is attributed to a compression RD problem [25].”
Hidden Markov processes (2002, background): “Composite sources with i.i.d. switch processes and finite number of statistically independent i.i.d. subsources were first introduced by Berger [31].”
Power-rate-distortion analysis for wireless video communication under energy constraints (2005, background): “…has been one of the major research focus in information theory and communication for the past several decades, from the early Shannon's source coding theorem for asymptotic R-D analysis of generic information data [6], to recent R-D modeling of modern video encoding systems [10], [12], [13], [22].”