Training highly multiclass classifiers

By Maya R. Gupta, Samy Bengio, Jason Aaron Edward Weston. 2014

100 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Quoc V. Le
2. Mohit Bansal
3. Yoshua Bengio
4. Percy Liang
5. Bernt Schiele
6. Jianfeng Gao
7. Richard Socher
8. Luke S. Zettlemoyer

In Evaluation & Benchmarks

1. Noam Shazeer
2. Mitchell Stern
3. Bernt Schiele
4. Matthias Hein
5. Maksim Lapin
6. Jason Wei
7. Francisco J. R. Ruiz
8. Jason Aaron Edward Weston

Citations in the corpus

• Adafactor: Adaptive Learning Rates with Sublinear Memory Cost (2018, methodology): “Also, Gupta et al. (2014) employ a similar technique, saving memory by averaging Adagrad accumulators across embedding vectors.”
• Generalized Low Rank Models (2014, background): “While one-vs-all classification performs about as well as more sophisticated loss functions on small data sets [123], these more sophisticated nonseparable loss tend to perform much better as the number of classes (and examples) increases [56].”
• Top-k Multiclass SVM (2015, methodology): “As the number of classes increases, two important issues emerge: class overlap and multilabel nature of examples [9].”
• #TagSpace: Semantic Embeddings from Hashtags (2014, result): “We expect performance of linear models such as this to be similar to WSABIE as that has been in the case in other datasets (Gupta et al., 2014), but at the cost of more memory usage.”
• Augment and Reduce: Stochastic Inference for Large Categorical Distributions (2018, background): “Examples include classification (Gupta et al., 2014), language models (Bengio et al.”
• Loss Functions for Top-k Error: Analysis and Insights (2015, background): “Top-k optimization has recently received revived attention with the advent of large scale problems [18, 23, 24, 25].”
• High-performance, Distributed Training of Large-scale Deep Learning Recommendation Models (2021, methodology): “The row-wise sparse AdaGrad was first introduced in [11], and then further elaborated in [56].”
• Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning (2021, background): “, 2017), however, have a converse setup characterized by a large number of output classes (Gupta et al., 2014), often with few training examples per class.”
• Extreme Tensoring for Low-Memory Preconditioning (2019, methodology): “Similar algorithms have appeared in prior work [GBW14, SMM + 17].”