Cover

2022

Citations in the corpus

• Shaking the foundations: delusions in sequence models for interaction and control (2021, background): “In fact, it is optimal in expectation with respect to the log-loss; and if the sequence is governed by latent variables (which is the case, for instance, when the sequence is exchangeable) then the scheme corresponds to Bayesian prediction [7].”
• Excessive Invariance Causes Adversarial Vulnerability (2018, background): “Consider the reformulation by the chain rule of mutual information (Cover & Thomas, 2006), where I ( y ; z n | z s ) denotes the conditional mutual information.”
• A Variational Perspective on Diffusion-Based Generative Models and Score Matching (2021, background): “This type of formulas fall into the category of de Bruijn’s identity (Cover, 1999) for relative entropy.”
• Information-Theoretic Probing for Linguistic Structure (2020, positive): “We make this statement more formal: We assert that the goal of probing ought to be estimating the mutual information (Cover and Thomas, 2012) be-tween a representation-valued random variable and a linguistic property-valued random variable.”
• What do you mean, BERT? Assessing BERT as a Distributional Semantics Model (2019, methodology): “Yet this methodology has been criticized as potentially conflicting with the intended purpose of studying the representations themselves (Wieting and Kiela, 2019; Cover, 1965); cf. also Hewitt and Liang (2019) for a discussion.”
• Conditional Contrastive Learning: Removing Undesirable Information in Self-Supervised Representations (2021, background): “We now discuss one idea to remove undesirable information from a variable from the self-supervised representations: conditioning on it [7].”
• ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification (2022, background): “An ideal encryption scheme causes the generated message to bear the maximum possible entropy [5].”
• Sustainable Post-Mining Land Use: Are Closed Metal Mines Abandoned or Re-Used Space? (2017, methodology): “Land cover information were derived from the CORINE Land Cover 2012 database with 25 m spatial resolution [68].”
• One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing (2020, methodology): “We further compress these values using an entropy encoder [14].”
• Intrinsic Evaluation of Summarization Datasets (2020, unknown): “I ( S ; M ) ≤ I ( S ; T, P, A ) (Cover and Thomas, 2006, Thm. 2.8.1) ≤ I ( S ; T ) + I ( S ; P ) + I ( S ; A ) (Duchi, 2019, Inequality 2.1.7)”
• An Information-Theoretic Approach to Transferability in Task Transfer Learning (2019, background): “See [18] for more background information on error exponents and its related theorems.”
• JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services (2018, background): “There are two general categories of compression techniques, lossy and lossless [5].”
• How to Train Your Energy-Based Models (2021, methodology): “…consequence of the connection between those two statistical divergences, as characterized by a relative version of the de Bruijin’s identity (Cover, 1999; Lyu, 2012): Here q t (˜ x ) and p θ ,t (˜ x ) denote smoothed versions of p data ( x ) and p θ ( x ) , resulting from adding Gaussian…”
• Preventing Posterior Collapse with delta-VAEs (2019, background): “…at other timesteps given the current one (i.e., p ( x t | z ) = p ( x t | z t ) ), there will then be at best (by the data processing inequality (Cover & Thomas, 2006)) a break-even situation between the KL cost of encoding information in z t and the resulting improvement in the reconstruction…”
• Caveats for information bottleneck in deterministic scenarios (2018, unknown): “We first bound the (cid:96) 1 distance between p Y and ˜ p Y , By (Cover & Thomas, 2012, Thm. 17.3.3), We now bound the magnitude of the difference of mutual informations as where in the last line we’ve used Eq.”
• A Bayesian Framework for Information-Theoretic Probing (2021, background): “2 Background: Information Theory Information theory (Shannon, 1948; Cover and Thomas, 2006) provides us with a number of tools to analyse data and their associated probability distributions—among which are the entropy and mutual information.”