Adaptive Relevance Matrices in Learning Vector Quantization

By Petra Schneider, Michael Biehl, Barbara Hammer. Neural Computation, 2009

415 citations recorded by the source corpus.

Abstract We propose a new matrix learning scheme to extend relevance learning vector quantization (RLVQ), an efficient prototype-based classification algorithm, toward a general adaptive metric. By introducing a full matrix of relevance factors in the distance measure, correlations between different features and their importance for the classification scheme can be taken into account and automated, and general metric adaptation takes place during training. In comparison to the weighted Euclidean metric used in RLVQ and its variations, a full matrix is more powerful to represent the internal structure of the data appropriately. Large margin generalization bounds can be transferred to this case, leading to bounds that are independent of the input dimensionality. This also holds for local metrics attached to each prototype, which corresponds to piecewise quadratic decision boundaries. The algorithm is tested in comparison to alternative learning vector quantization schemes using an artificial data set, a benchmark multiclass problem from the UCI repository, and a problem from bioinformatics, the recognition of splice sites for C. elegans.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/c7a1521262c43d26f8895ba1626ad5e66bad3528

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

CRITICAL ASSESSMENT OF AUTOMATED FLOW CYTOMETRY DATA ANALYSIS TECHNIQUES (2013, methodology): “A data driven classifier was obtained by means of a recently developed machine learning technique: Generalized Matrix Relevance Learning Vector Quantization (GMLVQ) provides easy to implement, highly flexible, intuitive classifiers and has proven to yield competitive performance in many practical pr”
Prototype-based models in machine learning. (2016, background): “to X i Λii = X i, j Ω(2)ij = 1 is imposed after each update step.(77) Cost-function-based schemes can be extended by an adaptive relevance matrix, analogously; see Refs 77 and 87.”
Incremental learning algorithms and applications (2016, background): “For high dimensional data, adaptive low-rank metric learning schemes can dramatically improve classification accuracy and efficiency [17, 145].”
Incremental on-line learning: A review and comparison of state of the art algorithms (2018, background): “Metric learning, as described in [38, 39], can also be applied to extend the classification abilities further.”
Prototype-based Neural Network Layers: Incorporating Vector Quantization (2018, background): “Generalized Matrix LVQ (GMLVQ [40]) supposes an adjustable dissimilarity measure, which is adapted during the training process.”
Support vector machines in remote sensing: A review (2011, neutral): “Inaddition,integrationwithmethodologies that deal more naturally with multi-class problems without the SVM complexity may further advance SVM understanding, for example a learning vector quantization system (Schneider et al., 2009).”