Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems

By Imre Csiszár. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Proper local scoring rules (2011, background): “It follows that H(P ) = ∫ dµ(x)φ{p(x)} and so, assuming H(P ) is finite, the corresponding (separable) Bregman divergence [Bregman (1967), Csiszár (1991)]—also termed U -divergence [Eguchi (2008)]—is d(P,Q) = ∫ dµ(x) ([φ{q(x)}+ {p(x)− q(x)}φ′{q(x)}]− φ{p(x)}).”
An Economic Evaluation of Empirical Exchange Rate Models (2009, unknown): “This is an attractive feature of our Bayesian approach because there is a complete axiomatic foundation that justi es why KLIC is the best criterion a rational agent should use in choosing between models (e.g. Csiszar, 1991).”
A QUARTET OF SEMIGROUPS FOR MODEL SPECIFICATION, ROBUSTNESS, PRICES OF RISK, AND MODEL DETECTION (2003, methodology): “As a measure of discrepancy, it has been axiomatized by Csiszar (1991) although his defense shall not concern us here.”
Accompanying document to "Point Estimation with Exponentially Tilted Empirical Likelihood" (2005, background): “Entropy maximization has a long history as a device to construct distributions which properly model lack of prior information under a set of known constraints (see, e.g., [ 12 , 17, 36, 38, 60]).”
Joint additive Kullback–Leibler residual minimization and regularization for linear inverse problems (2007, methodology): “In an examination of an axiomatic approach to inference for linear inverse problems, [24] compares the roles of least squares and entropy measures.”
Mathematics of Signal Processing (2004, background): “Unwilling to turn the problem over entirely to the philosophers, a number of mathematicians and physicists have sought mathematical solutions to this inference problem, framed in terms of what the most likely answer is, or which answer involves the smallest amount of additional prior information [90”
LQF: Linear Quadratic Fine-Tuning (2020, background): “This may seem counter-intuitive, but there is growing evidence that the MSE can be competitive to train classifiers [13, 3, 18] and is just as well grounded theoretically [5].”
Sticky information and model uncertainty in survey data on inflation expectations (2007, background): “VAR’s estimated every 12 (or 24) months will produce forecasts similar to the 9 month predictor and our estimation strategy will not be able to separately identify agents with a 9 or 12 month predictor.(18) It is important to note that this bound does not affect the qualitative results.”
Clustering with Bregman Divergences (2005, background): “which is exactly the Itakura-Saito distance between the power spectra F(e jθ) and G(e jθ) and can also be interpreted as the I-divergence (Csiszár, 1991) between the generating processes under the assumption that they are equal mean, stationary Gaussian processes (Kazakos and Kazakos, 1980).”
Richardson–Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution (2006, methodology): “To quantify the quality of the deconvolution we used the I-divergence criterion, which is the best measure in the presence of a non-negativity constraint (Csiszár, 1991).”
A Descent Lemma Beyond Lipschitz Gradient Continuity: First-Order Methods Revisited and Applications (2017, methodology): “A natural and very well-known measure of proximity of two nonnegative vectors is based on the so-called Kullback-Liebler divergence (relative entropy functional) see Csiszár [17],”