Comparison of classifier methods: a case study in handwritten digit recognition

By L. Eon Bottou, Corinna Cortes, John S. Denker, Harris Drucker, Isabelle M Guyon, L. Jackel, Yann LeCun, Urs Muller, Eduard Säckinger, Patrice Y. Simard and 1 more. 1994

This paper compares the performance of several classifier algorithms on a standard database of handwritten digits. We consider not only raw accuracy, but also training time, recognition time, and memory requirements. When available, we report measurements of the fraction of patterns that must be rejected so that the remaining patterns have misclassification rates less than a given threshold.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Invariant Risk Minimization (2019, background): “For instance, whereas the original NIST handwritten data was collected from different writers under different conditions [19], the popular MNIST training and testing sets [8] were carefully shuffled to represent similar mixes of writers.”
Fixed-point feedforward deep neural network design using weights +1, 0, and −1 (2014, methodology): “This database has been widely used for evaluation of various image classifiers [2], [15], [16].”
Hessian-based Analysis of Large Batch Training and Robustness to Adversaries (2018, methodology): “setup: For the MNIST experiments, we train a standard LeNet on MNIST dataset [3] (using M1 network).”
A comparison of methods for multiclass support vector machines (2002, methodology): “The earliest used implementation for SVM multiclass classification is probably the one-against-all method (for example, [2]).”
Recent Advances of Large-Scale Linear Classification (2012, methodology): “• One-against-rest method If there are k classes in the training data, the one-against-rest method [54] constructs k binary classification models.”
Using discriminant analysis for multi-class classification: an experimental investigation (2006, unknown): “Depending on the reduction technique that is used , the group can be further divided into one - versus - the - rest method ( Schölkopf & Smola , 2002 ; Bottou et al .”
SVM-Based Multimodal Classification of Activities of Daily Living in Health Smart Homes: Sensors, Algorithms, and First Experimental Results (2010, unknown): “The simplest manner to address this problem , called “One-versus-all” was to train N binary classifiers th at learned to distinguish one class from all the others [35].”
Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines (2007, methodology): “For facial expression recognition using multiclass SVMs, the grid deformation feature vector is used as an input to a multi class SVMs system [48].”
Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks (2020, background): “How to compare between diﬀerent benchmarks for the same task (e.g., COCO vs. MNIST (Bottou et al., 1994) vs. ImageNet (Deng et al., 2009)) or diﬀerent tasks for the same benchmark?”
Shape Quantization and Recognition with Randomized Trees (1997, background): “There may also be some morphological operations to standardize stroke thickness (Bottou et al., 1994; Hastie, Buja, & Tibshirani, 1995).”
Optical character recognition for self-service banking (1995, background): “AT&T TECHNICAL JOURNAL • JULY/AUGUST 1995 19 Label: P(2) = 1 Input Labeled training examples Neural network”