Continuous Learning of Context-dependent Processing in Neural Networks

By Guanxiong Zeng, Yang Chen, Bo Cui, Shan Yu. 2018

36 citations recorded by the source corpus.

Subfields: Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Meta & Continual Learning

1. Bing Liu
2. Zixuan Ke
3. Bin Liu
4. Gyuhak Kim
5. Dongyan Zhao
6. Mehrdad Farajtabar
7. Hu Xu
8. Seyed Iman Mirzadeh

Citations in the corpus

• A Theoretical Study on Solving Continual Learning (2022, background): “For orthogonal projection, we use OWM [49].”
• Adaptive Orthogonal Projection for Batch and Online Continual Learning (2022, methodology): “Architecture : Since AOP aims to improve OWM (Zeng et al. 2019), it uses the same architecture as OWM.”
• Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, methodology): “Zeng et al. (2019) tried to learn the new task by revising the weights in the orthogonal direction of the old task data.”
• Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning (2021, methodology): “For baselines, EWC, UCL, OWM and HAT, although they perform better than NFH, they are significantly poorer than CTR as they don’t have methods to encourage knowledge transfer for ASC and DSC. KAN and SRK do knowledge transfer but they are weaker than many other CL methods.”
• Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks (2021, methodology): “HAT, UCL, EWC and OWM were originally designed for image classification.”
• Continual Learning by Using Information of Each Class Holistically (2021, methodology): “For OWM, the model size is [InputSize-100* N ]-[100* N -”
• Gradient Projection Memory for Continual Learning (2021, methodology): “On the other hand, OWM reduces forgetting by modifying the weights of the network in the orthogonal to the input directions of the past tasks.”
• Linear Mode Connectivity in Multitask and Continual Learning (2020, background): “They can either a Bayesian (Nguyen et al., 2017; Titsias et al., 2019; Schwarz et al., 2018; Ebrahimi et al., 2019; Ritter et al., 2018) or frequentist views (Farajtabar et al., 2019; He & Jaeger, 2018; Zeng et al., 2018).”
• HRN: A Holistic Approach to One Class Learning (2020, methodology): “Specifically, a MLP of size [784-100]-[100-1] is used for MNIST and fMNIST, of size 3*[1024-300]-[900-300]-[300-1] for CIFAR-10 12 and of size [125-100]-[100-1] for KDDCUP99, and of size [6-100]-[100-1] for Thyroid.”
• Online Continual Learning through Mutual Information Maximization (2022, unknown): “Another approach is orthogonal projection (Zeng et al., 2019; Guo et al., 2022).”
• BNS: Building Network Structures Dynamically for Continual Learning (2021, background): “In the extreme, [62] finds orthogonal projections of weight updates that do not disturb the weights of old tasks.”