Cross-entropy vs. squared error training: a theoretical and experimental comparison

By Pavel Golik, Patrick Doetsch, Hermann Ney. 2013

247 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Mixed-Privacy Forgetting in Deep Networks (2020, methodology): “To train the user weights we use a mean square error (i.e., L 2 ) loss [23, 34, 17]: where y is a one-hot encoding of the class label.”
Multi-head CNN-RNN for multi-time series anomaly detection: An industrial case study (2019, methodology): “As architectures are designed for a binary classiﬁ-cation task, Binary cross-entropy [80] loss function is used.”
Large-Scale Transportation Network Congestion Evolution Prediction Using Deep Learning Theory (2015, methodology): “This is because that the cross-entropy indicates the distance between the probability distributions of computed outputs and target outputs, and thus it is preferable to using mean squared error for a neural network classifier with binary values [27].”
Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Tasks (2020, background): “ess of squared-error versus cross-entropy in estimating posterior probabilities with small neural networks, ﬁve or less nodes in each layer, and argued that cross-entropy had a performance advantage. Golik et al. (2013) provided a comparison of cross-entropy and squared error training for a hybrid H”
RASR/NN: The RWTH neural network toolkit for speech recognition (2014, methodology): “While the binary format is more efficient, the XML format makes it easy to debug and analyze models as well as to import models from other neural network software.”
NIMA: Neural Image Assessment (2017, background): “Yet, it has been shown that for ordered classes, the classification frameworks can outperform regression models [21], [31].”