One-Pass Learning with Incremental and Decremental Features
By Chenping Hou, Zhi-Hua Zhou. 2016
108 citations recorded by the source corpus.
In many real tasks the features are evolving, with some features vanished and some other features being augmented. For example, in environment monitoring some sensors expired whereas some new ones were deployed; in mobile game recommendation some games dropped whereas some new ones were added. Learning with such incremental and decremental features is crucial but rarely studied, particularly when the data comes like a stream and thus it is infeasible to keep the whole data for optimization. In this paper, we study this challenging problem and present the OPID approach. Our approach attempts to compress important information of vanished features into functions of survived features, and then expand to include the augmented features. It is an one-pass learning approach, which only needs to scan each instance once and does not need to store the whole data, and thus satisfies the evolving streaming data nature. After tackling this problem in one-shot scenario, we then extend it to multi-shot case. Empirical study on a broad range of data sets shows that our approach can address this problem effectively.
Subfields: Artificial Intelligence, Management Science and Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Management Science and Operations Research
Citations in the corpus
- One-Pass Learning via Bridging Orthogonal Gradient Descent and Recursive Least-Squares (2022, neutral): “…where the algorithm (i) makes an update for the current datapoint without direct access to previous data; (ii) the new updates do not significantly affect the predictions on the previous data; moreover, (iii) the computational and memory costs of each update must not grow with the iteration count.”
- Learning with Recoverable Forgetting (2022, positive): “In addition to the above works that tend to forbid the catastrophic forgetting of the old tasks, some researchers [19,64,18,17] pay more attention on the decremental cases where some features may vanish while feature evolving.”
- Learning With Feature Evolvable Streams (2017, background): “OPID [35] is the most related work to ours.”
- Learning with Feature and Distribution Evolvable Streams (2020, background): “Hou & Zhou (2018) assume the existence of overlapping features and design a “compress-then-expand”-style algorithm to exploit the knowledge in the evolving features.”
- Learnware: on the future of machine learning (2016, background): “, data distribution may change [3], new classes may occur [4,5], features may change [6], etc.”
- Small Sample Learning in Big Data Era (2018, background): “Furthermore, when we consider the data features/classes are both incremental and decremental, the setting is called online learning (Hou and Zhou, 2017).”
- Heterogeneous Few-Shot Model Rectification With Semantic Mapping (2020, background): “OPID [39] changes the training in the former task, and ensembles the last stage rectified classifier with stacking.”
- Π-ORFit: One-Pass Learning with Bregman Projection (2024, background): “The definition of one-pass learning is slightly different from single-pass online learning , which only aims to match the prediction accuracy of batch learners without executing multiple passes through the same training data [7]–[10].”
- Storage Fit Learning with Feature Evolvable Streams (2020, background): “Over the last several years, feature evolvable learning has drawn extensive attentions (Zhang et al. 2016; Hou, Zhang, and Zhou 2017a; Hou and Zhou 2018; Ye et al. 2018; Zhang et al. 2020), where old features will vanish and new features will emerge when data streams come continuously or in an onlin”
- IncDet: In Defense of Elastic Weight Consolidation for Incremental Object Detection (2020, background): “Broad learning system (BLS) [36], [37] develops incremental learning algorithms for increment of input variables [38], feature nodes [39] and enhancement nodes.”