Dynamic Feedback Engines: Layer-Wise Control for Self-Regulating Continual Learning
- URL: http://arxiv.org/abs/2512.21743v1
- Date: Thu, 25 Dec 2025 17:27:43 GMT
- Title: Dynamic Feedback Engines: Layer-Wise Control for Self-Regulating Continual Learning
- Authors: Hengyi Wu, Zhenyi Wang, Heng Huang,
- Abstract summary: We propose an entropy-aware continual learning method that employs a dynamic feedback mechanism to regulate each layer based on its entropy.<n>Our approach reduces entropy in high-entropy layers to mitigate underfitting and increases entropy in overly confident layers to alleviate overfitting.<n> Experiments on various datasets demonstrate substantial performance gains over state-of-the-art continual learning baselines.
- Score: 55.854208296248714
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Continual learning aims to acquire new tasks while preserving performance on previously learned ones, but most methods struggle with catastrophic forgetting. Existing approaches typically treat all layers uniformly, often trading stability for plasticity or vice versa. However, different layers naturally exhibit varying levels of uncertainty (entropy) when classifying tasks. High-entropy layers tend to underfit by failing to capture task-specific patterns, while low-entropy layers risk overfitting by becoming overly confident and specialized. To address this imbalance, we propose an entropy-aware continual learning method that employs a dynamic feedback mechanism to regulate each layer based on its entropy. Specifically, our approach reduces entropy in high-entropy layers to mitigate underfitting and increases entropy in overly confident layers to alleviate overfitting. This adaptive regulation encourages the model to converge to wider local minima, which have been shown to improve generalization. Our method is general and can be seamlessly integrated with both replay- and regularization-based approaches. Experiments on various datasets demonstrate substantial performance gains over state-of-the-art continual learning baselines.
Related papers
- Flexible Entropy Control in RLVR with Gradient-Preserving Perspective [19.86794452199207]
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a critical method for enhancing the reasoning capabilities of Large Language Models (LLMs)<n>This paper proposes reshaping entropy control in RL from the perspective of Gradient-Preserving Clipping.<n>We introduce a novel regulation mechanism using dynamic clipping threshold to precisely manage entropy.
arXiv Detail & Related papers (2026-02-10T13:42:12Z) - Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation [58.3773038915023]
Continual learning aims to adapt pre-trained models to sequential tasks without forgetting previously acquired knowledge.<n>Most existing approaches treat continual learning as avoiding interference with past updates, rather than considering what properties make the current task-specific update naturally preserve previously acquired knowledge.<n>We address this problem using a projected first-order method compatible with standard deep-dots used in vision-language models.
arXiv Detail & Related papers (2026-01-31T13:27:02Z) - Entropy-Guided Token Dropout: Training Autoregressive Language Models with Limited Domain Data [89.96277093034547]
We introduce EntroDrop, an entropy-guided token dropout method that functions as structured data regularization.<n>We show that EntroDrop consistently outperforms standard regularization baselines and maintains robust performance throughout extended multi-epoch training.
arXiv Detail & Related papers (2025-12-29T12:35:51Z) - Ranked Entropy Minimization for Continual Test-Time Adaptation [7.5140668729696145]
Test-time adaptation aims to adapt to realistic environments in an online manner by learning during test time.<n>Entropy minimization has emerged as a principal strategy for test-time adaptation due to its efficiency and adaptability.<n>We propose ranked entropy minimization to mitigate the stability problem of the entropy minimization method.
arXiv Detail & Related papers (2025-05-22T09:29:38Z) - Normalization and effective learning rates in reinforcement learning [52.59508428613934]
Normalization layers have recently experienced a renaissance in the deep reinforcement learning and continual learning literature.
We show that normalization brings with it a subtle but important side effect: an equivalence between growth in the norm of the network parameters and decay in the effective learning rate.
We propose to make the learning rate schedule explicit with a simple re- parameterization which we call Normalize-and-Project.
arXiv Detail & Related papers (2024-07-01T20:58:01Z) - Stabilizing Transformer Training by Preventing Attention Entropy
Collapse [56.45313891694746]
We investigate the training dynamics of Transformers by examining the evolution of the attention layers.
We show that $sigma$Reparam successfully prevents entropy collapse in the attention layers, promoting more stable training.
We conduct experiments with $sigma$Reparam on image classification, image self-supervised learning, machine translation, speech recognition, and language modeling tasks.
arXiv Detail & Related papers (2023-03-11T03:30:47Z) - Entropy-based Stability-Plasticity for Lifelong Learning [17.40355682488805]
We propose Entropy-based Stability-Plasticity (ESP) to address the stability-plasticity dilemma in neural networks.
Our approach can decide dynamically how much each model layer should be modified via a plasticity factor.
In some cases, it is possible to freeze layers during training leading to speed up in training.
arXiv Detail & Related papers (2022-04-18T22:58:49Z) - FOSTER: Feature Boosting and Compression for Class-Incremental Learning [52.603520403933985]
Deep neural networks suffer from catastrophic forgetting when learning new categories.
We propose a novel two-stage learning paradigm FOSTER, empowering the model to learn new categories adaptively.
arXiv Detail & Related papers (2022-04-10T11:38:33Z) - Partial local entropy and anisotropy in deep weight spaces [0.0]
We refine a recently-proposed class of local entropic loss functions by restricting the smoothening regularization to only a subset of weights.
The new loss functions are referred to as partial local entropies. They can adapt to the weight-space anisotropy, thus outperforming their isotropic counterparts.
arXiv Detail & Related papers (2020-07-17T16:16:18Z) - Regularizing Meta-Learning via Gradient Dropout [102.29924160341572]
meta-learning models are prone to overfitting when there are no sufficient training tasks for the meta-learners to generalize.
We introduce a simple yet effective method to alleviate the risk of overfitting for gradient-based meta-learning.
arXiv Detail & Related papers (2020-04-13T10:47:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.