Distribution-Free One-Pass Learning

By Peng Zhao, Xinqiang Wang, Siyu Xie, Lei Guo, Zhi-Hua Zhou. 2017

35 citations recorded by the source corpus.

In many large-scale machine learning applications, data are accumulated over time, and thus, an appropriate model should be able to update in an online style. In particular, it would be ideal to have a storage independent from the data volume, and scan each data item only once. Meanwhile, the data distribution usually changes during the accumulation procedure, making distribution-free one-pass learning a challenging task. In this paper, we propose a simple yet effective approach for this task, without requiring prior knowledge about the change, where every data item can be discarded once scanned. We also present a variant for high-dimensional situations, by exploiting compressed sensing to reduce computational and storage complexity. Theoretical analysis shows that our proposal converges under mild assumptions, and the performance is validated on both synthetic and real-world datasets.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Youngjae Min
2. Yuan Jiang
3. Peng Zhao
4. Hyo-Sang Shin
5. Namhoon Cho
6. Zhenyu Zhang
7. Zhi-Hua Zhou
8. Navid Azizan

Citations in the corpus

• One-Pass Learning via Bridging Orthogonal Gradient Descent and Recursive Least-Squares (2022, neutral): “…where the algorithm (i) makes an update for the current datapoint without direct access to previous data; (ii) the new updates do not significantly affect the predictions on the previous data; moreover, (iii) the computational and memory costs of each update must not grow with the iteration count.”
• Π-ORFit: One-Pass Learning with Bregman Projection (2024, methodology): “Further, [3] introduced a distribution-free one-pass method for lifelong learning under possibly time-varying true data distribution, akin to the discrete-time RLS with exponential forgetting factor.”
• Learning with Feature and Distribution Evolvable Streams (2020, background): “Typically, most previous works assume that the distribution of nearby data is closer to the new data, and thus plenty of approaches are proposed, e.g., the sliding window scheme (Kuncheva & Žliobait˙e, 2009) and the forgetting factor mechanisms (Klinkenberg, 2004; Zhao et al., 2019).”