An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems

By Deniz Erdoğmuş, José Carlos Príncipe. IEEE Transactions on Signal Processing, 2002

358 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/1c16f6e06c7b1ac5934af4a94e28a0bec90f6c1f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Statistical and Nonlinear Physics

Citations in the corpus

Causality detection based on information-theoretic approaches in time series analysis (2007, neutral): “[174,186,70,71,205,251].”
The Generalized Cross Entropy Method, with Applications to Probability Density Estimation (2011, background): “Keywords and phrases: Cross entropy, information theory, Monte Carlo simulation, statistical modeling, kernel smoothing, functional optimization, bandwidth selection, calculus of variations.”
Generalized correlation function: definition, properties, and application to blind equalization (2006, methodology): “Typically, Renyi’s entropy or some approximation to the Kullback–Leibler distance have been used as ITL cost functions and they have achieved excellent results on a number of problems, e.g., time-series prediction [9], blind-source separation [10], or equalization [11].”
Correntropy: Properties and Applications in Non-Gaussian Signal Processing (2007, methodology): “Successful engineering solutions from these methodologies rely heavily on the Gaussianity and linearity assumptions.”
An introductory survey of probability density function control (2019, methodology): “Since MEE criterion ensures that the estimation error has small uncertainty, it was used for supervised training of non-linear stochastic system by Erdogmus and Principe (2002); Principe (2010).”
Generalized Correntropy for Robust Adaptive Filtering (2015, background): “Typical examples include mixednorm [9][10], M-estimate cost [11][12], and error entropy [13][14][15][16][17][18].”
The design and analysis of information processing systems ] From Linear Adaptive Filtering to Nonlinear Information Processing (2009, methodology): “The use of minimum error entropy (MEE) criterion as an alternative to MSE in supervised parametric model training (e.g., neural networks) has been studied by the authors [16], [17].”
Correntropy: A Localized Similarity Measure (2006, methodology): “the cost function is still directly estimated from data with a Parzen kernel estimator [2], but extracts more information from the data structure for the adaptation process, and yields therefore solutions that are more accurate than MSE for non-Gaussian processes [3]-[5].”