An intuitive proof of the data processing inequality

By Normand J. Beaudry, Renato Renner. 2011

153 citations recorded by the source corpus.

The data processing inequality (DPI) is a fundamental feature of information theory. Informally it states that you cannot increase the information content of a quantum system by acting on it with a local physical operation. When the smooth min-entropy is used as the relevant information measure, then the DPI follows immediately from the definition of the entropy. The DPI for the von Neumann entropy is then obtained by specializing the DPI for the smooth min-entropy by using the quantum asymptotic equipartition property (QAEP). We provide a short proof of the QAEP and therefore obtain a self-contained proof of the DPI for the von Neumann entropy.

Subfields: Artificial Intelligence, Atomic and Molecular Physics, and Optics, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Atomic and Molecular Physics, and Optics

In Statistical and Nonlinear Physics

Citations in the corpus

Theoretical evidence for adversarial robustness through randomization: the case of the Exponential family (2019, background): “It is a well-known inequality from information theory which states that “post-processing cannot increase information” [10, 3].”
Online Continual Learning through Mutual Information Maximization (2022, background): “Due to the Markov chain X → f θ ( X ) → Φ( f θ ( X )) and the data processing inequality (Beaudry & Renner, 2011), we have So what we optimize directly is a lower bound for I ( X ; f θ ( X )) .”
Quantum Conditional Mutual Information and Approximate Markov Chains (2014, methodology): “The method we use to obtain these bounds is inspired by a recent approach [2] to prove strong subadditivity of the vonNeumann entropy (seeEq.”
A Principled Approach to Learning Stochastic Representations for Privacy in Deep Neural Inference (2020, background): “The first inequality follows from the data processing inequality (Beaudry & Renner, 2011).”
Mobile sensor data anonymization (2018, methodology): “Therefore the transformation U→ X→ X′ is Markovian [26] and, based on the data processing inequality [3], the following holds: I (U;X) ≥ I (U;X′).”
A Statistical Turing Test for Generative Models (2023, background): “(1) is a restatement of the information processing lemma [4].”