InTAct: Interval-based Task Activation Consolidation for Continual Learning

• URL: http://arxiv.org/abs/2511.17439v1
• Date: Fri, 21 Nov 2025 17:36:12 GMT
• Title: InTAct: Interval-based Task Activation Consolidation for Continual Learning
• Abstract summary: InTAct preserves functional behavior in shared layers without freezing parameters or storing past data.<n>In doing so, InTAct stabilizes the functional role of important neurons rather than directly restricting parameter values.<n>Across diverse domain-incremental benchmarks, InTAct consistently reduces representation drift and improves performance.
• Score: 8.228364777296195
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual learning aims to enable neural networks to acquire new knowledge without forgetting previously learned information. While recent prompt-based methods perform strongly in class-incremental settings, they remain vulnerable under domain shifts, where the input distribution changes but the label space remains fixed. This exposes a persistent problem known as representation drift. Shared representations evolve in ways that overwrite previously useful features and cause forgetting even when prompts isolate task-specific parameters. To address this issue, we introduce InTAct, a method that preserves functional behavior in shared layers without freezing parameters or storing past data. InTAct captures the characteristic activation ranges associated with previously learned tasks and constrains updates to ensure the network remains consistent within these regions, while still allowing for flexible adaptation elsewhere. In doing so, InTAct stabilizes the functional role of important neurons rather than directly restricting parameter values. The approach is architecture-agnostic and integrates seamlessly into existing prompt-based continual learning frameworks. By regulating representation changes where past knowledge is encoded, InTAct achieves a principled balance between stability and plasticity. Across diverse domain-incremental benchmarks, including DomainNet and ImageNet-R, InTAct consistently reduces representation drift and improves performance, increasing Average Accuracy by up to 8 percentage points over state-of-the-art baselines.

Related papers

• Relative Parameter Importance in Task-Agnostic Replay-Free Continual Learning [2.141079906482723]
  We introduce a novel measure, the relative parameter-importance, which measures the relative importance of each parameter with respect to both the current and past tasks.<n>Unlike existing methods, our approach allows the update of parameters with high past-task importance when they have low relative-importance.<n>We demonstrate improvements against state-of-the-art CL methods on both class-incremental and domain-incremental learning text classification problems.
  arXiv  Detail & Related papers  (2026-08-01T12:38:55Z)
• Janus-LoRA: A Balanced Low-Rank Adaptation for Continual Learning [122.06915301391616]
  Low-Rank Adaptation (LoRA) has emerged as a promising paradigm for Continual Learning.<n>We propose textbfJanus-LoRA, a framework that restores this balance through two novel components.
  arXiv  Detail & Related papers  (2026-05-27T13:53:14Z)
• Trust the Batch, On- or Off-Policy: Adaptive Policy Optimization for RL Post-Training [50.86545293331458]
  Reinforcement learning is structurally harder than supervised learning because the policy changes the data distribution it learns from.<n>We present a simple yet effective batch-adaptive objective that replaces fixed clipping with the normalized effective sample size of the policy ratios.
  arXiv  Detail & Related papers  (2026-05-12T16:44:47Z)
• Representation Finetuning for Continual Learning [21.63273394445167]
  We introduce Continual Representation Learning (CoRe), a novel framework that shifts the finetuning paradigm from weight space to representation space.<n>By constraining updates to a low-rank subspace, CoRe achieves exceptional parameter efficiency.<n>Our work introduces representation finetuning as a new, more effective and interpretable paradigm for continual learning.
  arXiv  Detail & Related papers  (2026-03-11T18:15:31Z)
• Learning with Preserving for Continual Multitask Learning [4.847042727427382]
  We introduce Learning with Preserving (LwP), a novel framework that shifts the focus from preserving task outputs to maintaining the shared representation space.<n>LwP not only mitigates catastrophic forgetting but also consistently outperforms state-of-the-art baselines in CMTL tasks.
  arXiv  Detail & Related papers  (2025-11-11T22:23:20Z)
• EKPC: Elastic Knowledge Preservation and Compensation for Class-Incremental Learning [53.88000987041739]
  Class-Incremental Learning (CIL) aims to enable AI models to continuously learn from sequentially arriving data of different classes over time.<n>We propose the Elastic Knowledge Preservation and Compensation (EKPC) method, integrating Importance-aware importance Regularization (IPR) and Trainable Semantic Drift Compensation (TSDC) for CIL.
  arXiv  Detail & Related papers  (2025-06-14T05:19:58Z)
• Dynamic Continual Learning: Harnessing Parameter Uncertainty for Improved Network Adaptation [0.0]
  We propose using parameter-based uncertainty to determine which parameters are relevant to a network's learned function.<n>We show improved Continual Learning performance for Average Test Accuracy and Backward Transfer metrics.
  arXiv  Detail & Related papers  (2025-01-18T19:58:53Z)
• Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding [55.107555305760954]
  We propose a conceptually simple yet effective method that attributes forgetting to layer-wise parameter overwriting and the resulting decision boundary distortion. Our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02x the base model.
  arXiv  Detail & Related papers  (2024-01-17T09:01:29Z)
• Learning a Low-Rank Feature Representation: Achieving Better Trade-Off between Stability and Plasticity in Continual Learning [20.15493383736196]
  In continual learning, networks confront a trade-off between stability and plasticity when trained on a sequence of tasks. We propose a novel training algorithm called LRFR to bolster plasticity without sacrificing stability. Using CIFAR-100 and TinyImageNet as benchmark datasets for continual learning, the proposed approach consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-12-14T08:34:11Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• Adaptive Cross Batch Normalization for Metric Learning [75.91093210956116]
  Metric learning is a fundamental problem in computer vision. We show that it is equally important to ensure that the accumulated embeddings are up to date. In particular, it is necessary to circumvent the representational drift between the accumulated embeddings and the feature embeddings at the current training iteration.
  arXiv  Detail & Related papers  (2023-03-30T03:22:52Z)
• Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation [39.97128550414934]
  We present a novel class incremental learning approach based on deep neural networks. It continually learns new tasks with limited memory for storing examples in the previous tasks. Our algorithm is based on knowledge distillation and provides a principled way to maintain the representations of old models.
  arXiv  Detail & Related papers  (2022-04-02T16:30:04Z)
• On Generalizing Beyond Domains in Cross-Domain Continual Learning [91.56748415975683]
  Deep neural networks often suffer from catastrophic forgetting of previously learned knowledge after learning a new task. Our proposed approach learns new tasks under domain shift with accuracy boosts up to 10% on challenging datasets such as DomainNet and OfficeHome.
  arXiv  Detail & Related papers  (2022-03-08T09:57:48Z)
• Self-Challenging Improves Cross-Domain Generalization [81.99554996975372]
  Convolutional Neural Networks (CNN) conduct image classification by activating dominant features that correlated with labels. We introduce a simple training, Self-Challenging Representation (RSC), that significantly improves the generalization of CNN to the out-of-domain data. RSC iteratively challenges the dominant features activated on the training data, and forces the network to activate remaining features that correlates with labels.
  arXiv  Detail & Related papers  (2020-07-05T21:42:26Z)
• Enabling Continual Learning with Differentiable Hebbian Plasticity [18.12749708143404]
  Continual learning is the problem of sequentially learning new tasks or knowledge while protecting previously acquired knowledge. catastrophic forgetting poses a grand challenge for neural networks performing such learning process. We propose a Differentiable Hebbian Consolidation model which is composed of a Differentiable Hebbian Plasticity.
  arXiv  Detail & Related papers  (2020-06-30T06:42:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.