Challenges in representation learning: A report on three machine learning contests

By Ian J. Goodfellow, Dumitru Erhan, Pierre Luc Carrier, Aaron C. Courville, Mehdi Mirza, Benjamin Hamner, William J. Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee and 18 more. 2013

2,020 citations recorded by the source corpus.

The ICML 2013 Workshop on Challenges in Representation Learning(1) focused on three challenges: the black box learning challenge, the facial expression recognition challenge, and the multimodal learning challenge. We describe the datasets created for these challenges and summarize the results of the competitions. We provide suggestions for organizers of future challenges and some comments on what kind of knowledge can be gained from machine learning competitions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Evaluation & Benchmarks

Citations in the corpus

Pylearn2: a machine learning research library (2013, background): “…in part due to the attention the library received after it was used to set the state of the art on several computer vision benchmark tasks [21], and in part due to many Kagglers starting to use the library after the baseline solutions to some Kaggle contests were provided in Pylearn2 format [20].”
Learning Transferable Visual Models From Natural Language Supervision (2021, methodology): “These datasets include MNIST, the Facial Expression Recognition 2013 dataset (Goodfellow et al., 2015), STL-10 (Coates et al., 2011), EuroSAT (Helber et al., 2019), the NWPURESISC45 dataset (Cheng et al., 2017), the German Traffic Sign Recognition Benchmark (GTSRB) dataset (Stallkamp et al., 2011),…”
CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP (2021, neutral): “, 2018, 2019), FER2013 (Goodfellow et al., 2013), FGVC-Aircraft (Maji et al.”
A survey on Image Data Augmentation for Deep Learning (2019, methodology): “Using the emotion recognition dataset, FER2013 [94], Facial Expression Recognition Database, they build a CNN classifier to recognize 7 different emotions: angry, disgust, fear, happy, sad, surprise, and neutral.”
Data Augmentation in Emotion Classification Using Generative Adversarial Networks (2017, methodology): “In our experiments, three benchmark datasets are tested: Facial Expression Recognition Database (FER2013) [10], Static Facial Expressions in the Wild (SFEW) [5] and Japanese Female Facial Expression (JAFFE) Database [18].”
Addressing Class Imbalance in Federated Learning (2020, methodology): “We choose four different datasets: MNIST, CIFAR10, Fer2013 (Goodfellow et al. 2013), and FEMNIST of LEAF benchmark (Caldas et al. 2018).”
Eavesdrop the Composition Proportion of Training Labels in Federated Learning (2019, neutral): “Fer2013 [40] There are nearly 37,000 records about skin lesions, and they are classiﬁed into 7 labels, i.e., 0=Melanocytic nevi, 1=Melanoma, 2=Benign keratosis-like lesions, 3=Basal cell carcinoma, 4=Actinic keratoses, 5=Vascular lesions, 6=Dermatoﬁbroma.”
Image based Static Facial Expression Recognition with Multiple Deep Network Learning (2015, neutral): “ISBN 978-1-4503-3912-4/15/11 ...$15.00.”
Learning Social Relation Traits from Face Images (2015, background): “To explicitly learn these factors, an ideal solution is to introduce two additional loss functions on top of xl and xr respectively, representing that not only the concatenation of xl and xr learns the relation traits, but each of them also learns the high-level factors of its corresponding face…”
TransFER: Learning Relation-aware Facial Expression Representations with Transformers (2021, background): “FERPlus [3] is extended from FER2013 [13] which is a large-scale dataset collected by APIs in the Google image search.”
Context-Aware Emotion Recognition Networks (2019, methodology): “Most existing datasets [10, 43] have focused on the human facial analysis, and thus they are inappropriate for context-aware emotion recogntion.”
Emotion Recognition in Speech using Cross-Modal Transfer in the Wild (2018, methodology): “This dataset comprises the images from the original FER dataset (≈ 35k images) [28] together with a more extensive set of”
Training deep networks for facial expression recognition with crowd-sourced label distribution (2016, background): “Consequently, crowd-sourced labels on emotions exhibit only 65± 5% accuracy, as reported for the original FER data set [12].”
Reliable Crowdsourcing and Deep Locality-Preserving Learning for Unconstrained Facial Expression Recognition (2019, methodology): “tition [22], the winner [67] was based on Deep Convolutional Neural Network (DCNN) plus SVM.”
DAGER: Deep Age, Gender and Emotion Recognition Using Convolutional Neural Network (2017, background): “FER-2013 [22] and EmotiW are among the popular ones.”