MSplit LBI: Realizing Feature Selection and Dense Estimation Simultaneously in Few-shot and Zero-shot Learning

By Bo Zhao, Xinwei Sun, Yanwei Fu, Yuan Yao, Yizhou Wang. 2018

23 citations recorded by the source corpus.

It is one typical and general topic of learning a good embedding model to efficiently learn the representation coefficients between two spaces/subspaces. To solve this task, $L_{1}$ regularization is widely used for the pursuit of feature selection and avoiding overfitting, and yet the sparse estimation of features in $L_{1}$ regularization may cause the underfitting of training data. $L_{2}$ regularization is also frequently used, but it is a biased estimator. In this paper, we propose the idea that the features consist of three orthogonal parts, \emph{namely} sparse strong signals, dense weak signals and random noise, in which both strong and weak signals contribute to the fitting of data. To facilitate such novel decomposition, \emph{MSplit} LBI is for the first time proposed to realize feature selection and dense estimation simultaneously. We provide theoretical and simulational verification that our method exceeds $L_{1}$ and $L_{2}$ regularization, and extensive experimental results show that our method achieves state-of-the-art performance in the few-shot and zero-shot learning.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Cheng Deng
2. Kun-Juan Wei
3. Xu Yang
4. Da Chen
5. Yuan He
6. Pravendra Singh
7. Vinay P. Namboodiri
8. Piyush Rai

Citations in the corpus

• Rectification-based Knowledge Retention for Continual Learning (2021, background): “The average H values are 10.2%, 36.73%, 38.03%, 36.73% and 42.48% for base, SFT, L1, L2 and LZSL [30] respectively.”
• Self-Supervised Learning for Few-Shot Image Classification (2019, background): “Zhao et al. [17] split the features to three orthogonal parts to improve the classification performance for few-short learning, allowing simultaneous feature selection and dense estimation.”
• Lifelong Zero-Shot Learning (2020, background): “Zero-Shot Learning [Socher et al. , 2013; Zhang et al. , 2017; Zhao et al. , 2018; Chen et al. , 2018 ] has become a popular research topic, which aims to recognize unseen classes without any labeled training data.”