Hyperparameter Optimization and Boosting for Classifying Facial Expressions: How good can a "Null" Model be?

By James Bergstra, David D. Cox. 2013

23 citations recorded by the source corpus.

One of the goals of the ICML workshop on representation and learning is to establish benchmark scores for a new data set of labeled facial expressions. This paper presents the performance of a "Null" model consisting of convolutions with random weights, PCA, pooling, normalization, and a linear readout. Our approach focused on hyperparameter optimization rather than novel model components. On the Facial Expression Recognition Challenge held by the Kaggle website, our hyperparameter optimization approach achieved a score of 60% accuracy on the test data. This paper also introduces a new ensemble construction variant that combines hyperparameter optimization with the construction of ensembles. This algorithm constructed an ensemble of four models that scored 65.5% accuracy. These scores rank 12th and 5th respectively among the 56 challenge participants. It is worth noting that our approach was developed prior to the release of the data set, and applied without modification; our strong competition performance suggests that the TPE hyperparameter optimization algorithm and domain expertise encoded in our Null model can generalize to new image classification data sets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Challenges in representation learning: A report on three machine learning contests (2013, neutral): “Keywords: representation learning, competition, dataset”
Deep learning for computational biology (2016, methodology): “Frameworks such as Spearmint (Snoek et al , 2012), Hyperopt (Bergstra & Cox, 2013) or SMAC (Hutter et al , 2011) allow to automatically explore the hyper-parameter space using Bayesian optimization.”
Towards an Empirical Foundation for Assessing Bayesian Optimization of Hyperparameters (2013, methodology): “It has been used succesfully in several papers beyond the one in which it was introduced [4, 21, 3].”
Bayesian Hyperparameter Optimization for Ensemble Learning (2016, methodology): “Bergstra and Cox (2013) applied hyperparameter optimization in a multi-stage approach akin to boosting in order to generate better representations of images.”
Beyond Manual Tuning of Hyperparameters (2015, methodology): “It has been used successfully in several papers beyond the one in which it was introduced [7, 8, 67].”