Retrofit: Continual Learning with Bounded Forgetting for Security Applications

• URL: http://arxiv.org/abs/2511.11439v1
• Date: Fri, 14 Nov 2025 16:07:03 GMT
• Title: Retrofit: Continual Learning with Bounded Forgetting for Security Applications
• Authors: Yiling He, Junchi Lei, Hongyu She, Shuo Shao, Xinran Zheng, Yiping Liu, Zhan Qin, Lorenzo Cavallaro,
• Abstract summary: We propose RETROFIT, a data retrospective-free continual learning method that achieves bounded forgetting for effective knowledge transfer.<n>To mitigate interference, we apply low-rank and sparse updates that confine parameter changes to independent subspaces.<n>In malware detection under temporal drift, it substantially improves the retention score, from 20.2% to 38.6% over CL baselines, and exceeds the oracle upper bound on new data.
• Score: 25.185616916987158
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern security analytics are increasingly powered by deep learning models, but their performance often degrades as threat landscapes evolve and data representations shift. While continual learning (CL) offers a promising paradigm to maintain model effectiveness, many approaches rely on full retraining or data replay, which are infeasible in data-sensitive environments. Moreover, existing methods remain inadequate for security-critical scenarios, facing two coupled challenges in knowledge transfer: preserving prior knowledge without old data and integrating new knowledge with minimal interference. We propose RETROFIT, a data retrospective-free continual learning method that achieves bounded forgetting for effective knowledge transfer. Our key idea is to consolidate previously trained and newly fine-tuned models, serving as teachers of old and new knowledge, through parameter-level merging that eliminates the need for historical data. To mitigate interference, we apply low-rank and sparse updates that confine parameter changes to independent subspaces, while a knowledge arbitration dynamically balances the teacher contributions guided by model confidence. Our evaluation on two representative applications demonstrates that RETROFIT consistently mitigates forgetting while maintaining adaptability. In malware detection under temporal drift, it substantially improves the retention score, from 20.2% to 38.6% over CL baselines, and exceeds the oracle upper bound on new data. In binary summarization across decompilation levels, where analyzing stripped binaries is especially challenging, RETROFIT achieves around twice the BLEU score of transfer learning used in prior work and surpasses all baselines in cross-representation generalization.

Related papers

• ACU: Analytic Continual Unlearning for Efficient and Exact Forgetting with Privacy Preservation [39.0731790601695]
  Continual Unlearning (CU) aims to sequentially forget particular knowledge acquired during the Continual Learning phase.<n>Most existing unlearning methods require access to the retained dataset for re-training or fine-tuning.<n>We propose a novel gradient-free method for CU, named Analytic Continual Unlearning (ACU), for efficient and exact forgetting with historical data privacy preservation.
  arXiv  Detail & Related papers  (2025-05-18T05:28:18Z)
• CSTA: Spatial-Temporal Causal Adaptive Learning for Exemplar-Free Video Class-Incremental Learning [62.69917996026769]
  A class-incremental learning task requires learning and preserving both spatial appearance and temporal action involvement.<n>We propose a framework that equips separate adapters to learn new class patterns, accommodating the incremental information requirements unique to each class.<n>A causal compensation mechanism is proposed to reduce the conflicts during increment and memorization for between different types of information.
  arXiv  Detail & Related papers  (2025-01-13T11:34:55Z)
• DESIRE: Dynamic Knowledge Consolidation for Rehearsal-Free Continual Learning [23.878495627964146]
  Continual learning aims to equip models with the ability to retain previously learned knowledge like a human.<n>Existing methods usually overlook the issue of information leakage caused by the fact that the experiment data have been used in pre-trained models.<n>In this paper, we propose a new LoRA-based rehearsal-free method named DESIRE.
  arXiv  Detail & Related papers  (2024-11-28T13:54:01Z)
• Temporal-Difference Variational Continual Learning [77.92320830700797]
  We propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations.<n>Our approach effectively mitigates Catastrophic Forgetting, outperforming strong Variational CL methods.
  arXiv  Detail & Related papers  (2024-10-10T10:58:41Z)
• Beyond Prompt Learning: Continual Adapter for Efficient Rehearsal-Free Continual Learning [22.13331870720021]
  We propose a beyond prompt learning approach to the RFCL task, called Continual Adapter (C-ADA) C-ADA flexibly extends specific weights in CAL to learn new knowledge for each task and freezes old weights to preserve prior knowledge. Our approach achieves significantly improved performance and training speed, outperforming the current state-of-the-art (SOTA) method.
  arXiv  Detail & Related papers  (2024-07-14T17:40:40Z)
• Decision Boundary-aware Knowledge Consolidation Generates Better Instance-Incremental Learner [41.462673126500974]
  Instance-incremental learning (IIL) focuses on learning continually with data of the same classes. We propose a novel decision boundary-aware distillation method with consolidating knowledge to teacher to ease the student learning new knowledge.
  arXiv  Detail & Related papers  (2024-06-05T08:49:51Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization [89.54947228958494]
  This paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks. We propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework. TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness.
  arXiv  Detail & Related papers  (2023-03-20T14:12:55Z)
• New Insights on Reducing Abrupt Representation Change in Online Continual Learning [69.05515249097208]
  We focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream. We show that applying Experience Replay causes the newly added classes' representations to overlap significantly with the previous classes. We propose a new method which mitigates this issue by shielding the learned representations from drastic adaptation to accommodate new classes.
  arXiv  Detail & Related papers  (2022-03-08T01:37:00Z)
• Incremental Object Detection via Meta-Learning [77.55310507917012]
  We propose a meta-learning approach that learns to reshape model gradients, such that information across incremental tasks is optimally shared. In comparison to existing meta-learning methods, our approach is task-agnostic, allows incremental addition of new-classes and scales to high-capacity models for object detection.
  arXiv  Detail & Related papers  (2020-03-17T13:40:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.