Probabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition

By John Scott Bridle. 1989

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Flamingo: a Visual Language Model for Few-Shot Learning (2022, methodology): “We use a learnable temperature parameter in the final log-softmax layer [9].”
RNADE: The real-valued neural autoregressive density-estimator (2013, background): “The softmax [18] ensures the mixing fractions are positive and sum to one, the exponential ensures the standard deviations are positive.”
BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model (2019, methodology): “It is relatively straightforward to compute, as it is simply softmax normalization over M terms (Bridle, 1990).”
Hierarchical Mixtures of Experts and the EM Algorithm (1993, methodology): “Assuming identical covariance matrices of the form 2 I for each of the experts yields the following hierarchical probability model: g jji e (cid:0) 1 2(cid:27)2 (y(cid:0)(cid:22)ij) T (y(cid:0)(cid:22)ij) : Example (binary classi(cid:12)cation) In binary classi(cid:12)cation problems the output y…”
From image-level to pixel-level labeling with Convolutional Networks (2014, methodology): “Assuming an aggregation procedure aggreg() is chosen, we interpret imagelevel class scores as class conditional probabilities by applying a softmax [3]:”
Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers (2016, methodology): “Given a fixed-dimensional input from the lower layer, the classification layer affinetransforms it followed by a softmax activation function (Bridle, 1990) to compute the predictive probabilities for all the categories.”
From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification (2016, background): “…is a key component of several statistical learning models, encompassing multinomial logistic regression (McCullagh & Nelder, 1989), action selection in reinforcement learning (Sutton & Barto, 1998), and neural networks for multi-class classiﬁcation (Bridle, 1990; Goodfellow et al., 2016).”
Estimating phoneme class conditional probabilities from raw speech signal using convolutional neural networks (2013, background): “The Softmax [16] layer interprets network output scores fi(x) as conditional probabilities, for each class label i:”
Analysis of CNN-based speech recognition system using raw speech as input (2015, methodology): “It outputs the conditional probabilities p(i|x) for each class i, for each frame x using a SoftMax layer [19].”
Learning Distributed Representations of Concepts Using Linear Relational Embedding (2001, methodology): “Learning Distributed Representations of Concepts using Linear Relational Embedding Alberto Paccanaro Geoffrey Hinton Gatsby Unit”
Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning (2017, background): “predictions) by deep neural networks can be interpreted as the confidence of the network in that prediction (14).”
Sigsoftmax: Reanalysis of the Softmax Bottleneck (2018, methodology): “From these properties, we can regard output of the softmax trained by minimizing negative log-likelihood as a probability [4, 21].”
Bayesian Backpropagation Over I-O Functions Rather Than Weights (1993, background): “Example I: In schemes like softroax (Bridle, 1990), wdirectly parameterizes a conditional distribution f, with the output neuron index giving y.”
Sparse Sequence-to-Sequence Models (2019, background): “However, conventional seq2seq models are dense: they compute both attention weights and output probabilities with the softmax function (Bridle, 1990), which always returns positive values.”