Learning What Data to Learn

By Yang Fan, Fei Tian, Tao Qin, Jiang Bian, Tie-Yan Liu. 2017

85 citations recorded by the source corpus.

Machine learning is essentially the sciences of playing with data. An adaptive data selection strategy, enabling to dynamically choose different data at various training stages, can reach a more effective model in a more efficient way. In this paper, we propose a deep reinforcement learning framework, which we call \emph{\textbf{N}eural \textbf{D}ata \textbf{F}ilter} (\textbf{NDF}), to explore automatic and adaptive data selection in the training process. In particular, NDF takes advantage of a deep neural network to adaptively select and filter important data instances from a sequential stream of training data, such that the future accumulative reward (e.g., the convergence speed) is maximized. In contrast to previous studies in data selection that is mainly based on heuristic strategies, NDF is quite generic and thus can be widely suitable for many machine learning tasks. Taking neural network training with stochastic gradient descent (SGD) as an example, comprehensive experiments with respect to various neural network modeling (e.g., multi-layer perceptron networks, convolutional neural networks and recurrent neural networks) and several applications (e.g., image classification and text understanding) demonstrate that NDF powered SGD can achieve comparable accuracy with standard SGD process by using less data and fewer iterations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Forward and Reverse Gradient-Based Hyperparameter Optimization (2017, background): “(5)We note that a related approach based on reinforcement learning is presented in (Fan et al., 2017).”
An Empirical Study of Example Forgetting during Deep Neural Network Learning (2018, background): “Techniques of note — among others — are predeﬁned curricula of examples (Bengio & LeCun, 2007), self-paced learning (Kumar et al., 2010), and more recently meta-learning (Fan et al., 2017).”
Learning to select data for transfer learning with Bayesian Optimization (2017, methodology): “Other related work in this direction include using Re-inforcement Learning to learn what data to select during neural network training (Fan et al., 2017).”
Not All Samples Are Created Equal: Deep Learning with Importance Sampling (2018, background): “In addition, Fan et al. (2017) use reinforcement learning to train a neural network that selects samples for another neural network in order to optimize the convergence speed.”
Zero-shot Text Classification via Reinforced Self-training (2020, methodology): “Earlier, (Fan et al., 2017) proposed an adaptive data selection strategy, enabling to dynamically choose different data at different train-Figure ing stages.”
Attention in Natural Language Processing (2019, background): “While attention-less adaptive data selection strategies have already proven to be useful for efficiently obtaining more effective models [117], to the best of our knowledge, no attention-based approach has been experimented to date.”