Horizontal and Vertical Ensemble with Deep Representation for Classification
By Jingjing Xie, Bing Xu, Chuang Zhang. 2013
81 citations recorded by the source corpus.
Representation learning, especially which by using deep learning, has been widely applied in classication. However, how to use limited size of labeled data to achieve good classication performance with deep neural network, and how can the learned features further improve classication remain indenite. In this paper, we propose Horizontal Voting Vertical Voting and Horizontal Stacked Ensemble methods to improve the classication performance of deep neural networks. In the ICML 2013 Black Box Challenge, via using these methods independently, Bing Xu achieved 3rd in public leaderboard, and 7th in private leaderboard; Jingjing Xie achieved 4th in public leaderboard, and 5th in private leaderboard.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Andrew Gordon Wilson
- Pavel Izmailov
- Dmitry P. Vetrov
- Quoc V. Le
- Caiming Xiong
- Yoshua Bengio
- Richard Socher
- Gao Huang
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Bernt Schiele
- Dacheng Tao
- Xiaogang Wang
- Alan L. Yuille
- Trevor Darrell
- Ming-Hsuan Yang
- Jitendra Malik
Citations in the corpus
- Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs (2018, background): “[21] proposed a related ensembling approach that gathers outputs of neural networks from different epochs at the end of training to stabilize final predictions.”
- BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning (2020, background): “Ensembles of neural networks benefit from the above observation to achieve better performance by averaging or majority voting on the output of each ensemble member (Xie et al., 2013; Huang et al., 2017).”
- Snapshot Ensembles: Train 1, get M for free (2017, background): “Perhaps most similar to our work is that of Swann & Allinson (1998) and Xie et al. (2013), who explore creating ensembles from slices of the learning trajectory.”
- Ensemble deep learning for regression and time series forecasting (2014, methodology): “2) By setting the back propagation epochs from 100 to 2000 with step size equal to 100, we are able to get 20 prediction outputs y1 to y20 [21].”
- Ensemble deep learning: A review (2021, background): “Some attempts, like horizontal vertical voting of deep ensembles [118] have been made to obtain ensembles of deep models without independent training.”
- Ensemble deep learning in bioinformatics (2020, background): “…extracting intermediate features learned by the network to generate additional output for integration and thus stabilizing the ensemble prediction 99 ; and encouraging cooperation among individual models through a pairwise loss, thereby reducing the variance caused by small sample size 100 .”