Statistical Theory of Extreme Values and Some Practical Applications : A Series of Lectures

By E. Gumbel. 1954

Subfields: Statistics and Probability, Mathematical Optimization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Mathematical Optimization

Citations in the corpus

wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations (2020, methodology): “The Gumbel softmax enables choosing discrete codebook entries in a fully differentiable way [16, 24, 35].”
vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations (2019, methodology): “Given the dense representation z , we apply a linear layer, followed by a ReLU and another linear which outputs l ∈ R V logits for the Gumbel-Softmax.”
Categorical Reparameterization with Gumbel-Softmax (2016, methodology): “The Gumbel-Max trick (Gumbel, 1954; Maddison et al., 2014) provides a simple and efﬁcient way to draw samples z from a categorical distribution with class probabilities π : where g 1 ...g k are i.i.d samples drawn from Gumbel (0 , 1) 1 .”
A* Sampling (2014, background): “1 is known as max-stability—the highest order statistic of a sample of independent Gumbels also has a Gumbel distribution with a location that is the log partition function [13].”
The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables (2016, methodology): “This distribution features in extreme value theory (Gumbel, 1954) where it plays a central role similar to the Normal distribution: the Gumbel distribution is stable under max operations, and for some distributions, the order statistics (suitably normalized) of i.i.d. draws approach the Gumbel in…”
Adversarial Filters of Dataset Biases (2020, methodology): “The Gumbel method provides an efﬁcient way to perform such sampling (Gumbel & Lieblein, 1954; Mad-dison et al., 2014; Kim et al., 2016; Balog et al., 2017; Kool et al., 2019), by independently perturbing each ˜ p ( i ) with a Gumbel random variable and identifying k instances with the highest…”
AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks (2020, neutral): “During the training, the width of each layer is sampled through Gumbel-Softmax based on γ and the output is the weighted sum of the results of all operators based on the softmax value of α .”
Learning latent representations across multiple data domains using Lifelong VAEGAN (2020, methodology): “le the discrete latent variables a and c from f "() and f (), respectively, because the categorical representations are non-dierentiable. In order to mitigate this, we use the Gumbel-Max trick [23,41] for achieving the dierentiable relaxation of discrete random variables. The Gumbel-softmax tri”
Scalable Recollections for Continual Lifelong Learning (2017, methodology): “The Gumbel-Softmax function leverages the Gumbel-Max trick (Gumbel, 1954; Maddison et al., 2014) which provides an efﬁcient way to draw samples z from a categorical distribution with class probabilities p i representing the output of the encoder: In equation 1, g i is a sample drawn from…”
Boundary-Seeking Generative Adversarial Networks (2017, methodology): “The ”Gumbel technique“ (Gumbel & Lieblein, 1954; Jang et al., 2016) has been shown to be successful in some tasks where back-propogation through discrete ran-Table 1.”
Adversarial Feature Matching for Text Generation (2017, methodology): “Here we consider a soft-argmax operator (Zhang et al., 2016), similar to the Gumbel-softmax (Gumbel & Lieblein, 1954; Jang et al., 2017), when performing learning, as an approximation to (7): where (cid:12) represents the element-wise product.”
Deconvolutional Paragraph Representation Learning (2017, background): “where Dcos(x,y) is the cosine similarity defined as, 〈x,y〉 ||x||||y|| , We[v] is the v-th column of We, x̂ is the t-th column of X̂, τ is a positive number we denote as Gumbel-softmax parameter [27].”
Convolutional Networks with Adaptive Inference Graphs (2017, methodology): “We choose the Gumbel distribution for the noise, because of its key property that is known as the GumbelMax trick [9].”
Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement (2019, methodology): “The Gumbel-Max trick (Gumbel, 1954; Maddison et al., 2014) allows to sample from the categorical distribution (1) by independently perturbing the log-probabilities φ i with Gumbel noise and ﬁnding the largest element.”
Reparameterization Gradient for Non-differentiable Models (2018, methodology): “By applying the Gumbel-Max reparameterization [9, 21] to z, we transform p(x, z) to p(x, z, c), where c is sampled from the Gumbel distribution and z in p(x, z, c) is defined determin-”
Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundary (2017, unknown): “d samples drawn from Gumbel(0, 1) (Gumbel and Lieblein 1954).”
Learning joint latent representations based on information maximization (2021, unknown): “; yKg of a sample s by using the Gumble-Softmax trick, [31,32]: yt 1⁄4 expððlog st þ gtÞ=TÞ PK i expððlog si þ giÞ=TÞ ; ð8Þ”
Deep Mixture Generative Autoencoders (2021, methodology): “The Gumbel-softmax trick [44], [45] adopts the softmax function as a continuous, differentiable approximation to the argmax expression as [43] where m = { m 1 , m 2 , . . . , m K } is a K -dimensional masking vector and T is the temperature parameter.”
An end-to-end approach for speeding up neural network inference (2018, background): “However, Gumbel-Softmax (GS) [2, 10, 23, 34] provides a way to address this challenge.”