MedPEFT-CL: Dual-Phase Parameter-Efficient Continual Learning with Medical Semantic Adapter and Bidirectional Memory Consolidation

• URL: http://arxiv.org/abs/2511.17668v1
• Date: Fri, 21 Nov 2025 03:39:48 GMT
• Title: MedPEFT-CL: Dual-Phase Parameter-Efficient Continual Learning with Medical Semantic Adapter and Bidirectional Memory Consolidation
• Abstract summary: Medical vision-language segmentation models suffer from catastrophic forgetting when adapting to new anatomical structures.<n>We propose MedPEFT-CL, a parameter-efficient continual learning framework that addresses both efficient learning of new tasks and preservation of previous knowledge.<n>Our key contributions are: (1) a semantic-driven adapter allocation mechanism that enables efficient learning of new medical tasks, (2) a bi-modal LoRA adaptation that significantly reduces trainable parameters while maintaining cross-modal learning, and (3) bidirectional Fisher-memory coordination that prevents catastrophic forgetting from previous medical tasks.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical vision-language segmentation models suffer from catastrophic forgetting when adapting to new anatomical structures, requiring complete retraining that limits their clinical deployment. Although continual learning approaches have been studied for various applications, targeted research on continual learning approaches specifically designed for medical vision-language tasks remains underexplored. We propose MedPEFT-CL, a parameter-efficient continual learning framework that addresses both efficient learning of new tasks and preservation of previous knowledge through a dual-phase architecture based on CLIPSeg. Our dual-phase architecture features an adaptive learning phase that employs semantic similarity-based adapter allocation and parameter-efficient fine-tuning for medical tasks through prompt similarity analysis, and a knowledge consolidation phase employing bi-directional Fisher-memory coordination. This creates a reinforcing cycle: consolidation directs replay priorities while new tasks provide challenging samples that improve retention strategies. Our key contributions are: (1) a semantic-driven adapter allocation mechanism that enables efficient learning of new medical tasks, (2) a bi-modal LoRA adaptation that significantly reduces trainable parameters while maintaining cross-modal learning, and (3) bidirectional Fisher-memory coordination that prevents catastrophic forgetting from previous medical tasks. Extensive experiments across diverse medical datasets demonstrate superior forgetting mitigation and performance retention with minimal parameter overhead, making the framework effective for continual learning in medical vision-language scenarios.

Related papers

• An Empirical Analysis of Continual Learning for Heterogeneous Medical Visual Question Answering [2.0240196957224703]
  Deploying medical visual question answering systems in real-world clinical settings requires models that adapt to new clinical tasks without forgetting previously acquired knowledge. Continual learning (CL) provides a practical framework for this setting.<n>This work presents a systematic evaluation of CL for MedVQA across diverse clinical objectives, including classification, multi-label classification, detection, cell counting, and report generation.<n>Specifically, we explore (1) the ability of existing CL methods to mitigate catastrophic forgetting; (2) their sensitivity to task ordering, analyzing how different task sequences influence performance retention and forgetting; and (3) the evolution of low-rank adaptation parameters as new tasks are learned
  arXiv  Detail & Related papers  (2026-07-13T18:08:26Z)
• CoRE: Concept-Reasoning Expansion for Continual Brain Lesion Segmentation [16.903732775774124]
  Concept-Reasoning Expansion (CoRE) framework establishes a joint decision-making mechanism by integrating visual features with structured concepts.<n>CoRE achieves state-of-the-art performance and provides a high knowledge starting point for efficient future adaptation.
  arXiv  Detail & Related papers  (2026-04-28T08:39:43Z)
• Bi-CRCL: Bidirectional Conservative-Radical Complementary Learning with Pre-trained Foundation Models for Class-incremental Medical Image Analysis [35.9133156067447]
  Class-incremental learning (CIL) in medical image-guided diagnosis requires retaining prior diagnostic knowledge while adapting to newly emerging disease categories.<n>We propose Bi Conservative-Radical Complementary Learning (Bi-CRCL), a dual-learner framework inspired by complementary learning systems.<n> Experiments on five medical imaging datasets demonstrate consistent improvements over state-of-the-art methods under diverse settings.
  arXiv  Detail & Related papers  (2026-03-24T21:25:05Z)
• Duala: Dual-Level Alignment of Subjects and Stimuli for Cross-Subject fMRI Decoding [37.36610841961446]
  Cross-subject visual decoding aims to reconstruct visual experiences from brain activity across individuals.<n>Existing methods often suffer from degraded performance when adapting to new subjects with limited data.<n>We propose Duala, a dual-level alignment framework designed to achieve stimulus-level consistency and subject-level alignment.
  arXiv  Detail & Related papers  (2026-03-08T13:22:20Z)
• Fully Differentiable Bidirectional Dual-Task Synergistic Learning for Semi-Supervised 3D Medical Image Segmentation [3.9950415168730107]
  Semi-supervised learning relaxes the need of large pixel-wise labeled datasets for image segmentation by leveraging unlabeled data.<n>We propose a fully Differentiable Bidirectional Synergistic Learning (DBiSL) framework, which seamlessly integrates and enhances four critical SSL components.
  arXiv  Detail & Related papers  (2026-02-10T03:44:24Z)
• MedSeqFT: Sequential Fine-tuning Foundation Models for 3D Medical Image Segmentation [55.37355146924576]
  MedSeqFT is a sequential fine-tuning framework for medical image analysis.<n>It adapts pre-trained models to new tasks while refining their representational capacity.<n>It consistently outperforms state-of-the-art fine-tuning strategies.
  arXiv  Detail & Related papers  (2025-09-07T15:22:53Z)
• Baichuan-M2: Scaling Medical Capability with Large Verifier System [40.86227022086866]
  We introduce a novel dynamic verification framework that moves beyond static answer verifier.<n>We develop Baichuan-M2, a medical augmented reasoning model trained through a multi-stage reinforcement learning strategy.<n> evaluated on HealthBench, Baichuan-M2 outperforms all other open-source models and most advanced closed-source counterparts.
  arXiv  Detail & Related papers  (2025-09-02T11:23:35Z)
• NEARL-CLIP: Interacted Query Adaptation with Orthogonal Regularization for Medical Vision-Language Understanding [51.63264715941068]
  textbfNEARL-CLIP (iunderlineNteracted quunderlineEry underlineAdaptation with ounderlineRthogonaunderlineL Regularization) is a novel cross-modality interaction VLM-based framework.
  arXiv  Detail & Related papers  (2025-08-06T05:44:01Z)
• METHOD: Modular Efficient Transformer for Health Outcome Discovery [0.25112747242081457]
  This paper introduces METHOD, a novel transformer architecture specifically designed to address the challenges of clinical sequence modelling in electronic health records.<n>METHODintegrates three key innovations: (1) a patient-aware attention mechanism that prevents information leakage whilst enabling efficient batch processing; (2) an adaptive sliding window attention scheme that captures multi-scale temporal dependencies; and (3) a U-Net inspired architecture with dynamic skip connections for effective long sequence processing.<n> Evaluations on the MIMIC-IV database demonstrate that METHODconsistently outperforms the state-of-the-art ETHOSmodel
  arXiv  Detail & Related papers  (2025-05-16T15:52:56Z)
• Repurposing Foundation Model for Generalizable Medical Time Series Classification [16.21546283978257]
  FORMED is a framework for repurposing a backbone foundation model to enable highly generalizable MedTS classification on unseen datasets.<n>We evaluate FORMED on 5 diverse MedTS datasets, benchmarking against 11 Task-Specific Models (TSM) and 4 Task-Specific Adaptation (TSA) methods.<n>Our results demonstrate FORMED's dominant performance, achieving up to 35% absolute improvement in F1-score (on ADFTD dataset) over specialized baselines.
  arXiv  Detail & Related papers  (2024-10-03T23:50:04Z)
• Taxonomy Adaptive Cross-Domain Adaptation in Medical Imaging via Optimization Trajectory Distillation [73.83178465971552]
  The success of automated medical image analysis depends on large-scale and expert-annotated training sets. Unsupervised domain adaptation (UDA) has been raised as a promising approach to alleviate the burden of labeled data collection. We propose optimization trajectory distillation, a unified approach to address the two technical challenges from a new perspective.
  arXiv  Detail & Related papers  (2023-07-27T08:58:05Z)
• Adapter Learning in Pretrained Feature Extractor for Continual Learning of Diseases [66.27889778566734]
  Currently intelligent diagnosis systems lack the ability of continually learning to diagnose new diseases once deployed. In particular, updating an intelligent diagnosis system with training data of new diseases would cause catastrophic forgetting of old disease knowledge. An adapter-based Continual Learning framework called ACL is proposed to help effectively learn a set of new diseases.
  arXiv  Detail & Related papers  (2023-04-18T15:01:45Z)
• Leveraging Old Knowledge to Continually Learn New Classes in Medical Images [16.730335437094592]
  We focus on how old knowledge can be leveraged to learn new classes without catastrophic forgetting. Our solution is able to achieve superior performance over state-of-the-art baselines in terms of class accuracy and forgetting.
  arXiv  Detail & Related papers  (2023-03-24T02:10:53Z)
• MIMO: Mutual Integration of Patient Journey and Medical Ontology for Healthcare Representation Learning [49.57261599776167]
  We propose an end-to-end robust Transformer-based solution, Mutual Integration of patient journey and Medical Ontology (MIMO) for healthcare representation learning and predictive analytics.
  arXiv  Detail & Related papers  (2021-07-20T07:04:52Z)
• One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video [71.43912903508765]
  MDAL is a dynamic online adaptive learning scheme for instrument segmentation in robot-assisted surgery. It learns the general knowledge of instruments and the fast adaptation ability through the video-specific meta-learning paradigm. It outperforms other state-of-the-art methods on two datasets.
  arXiv  Detail & Related papers  (2021-03-24T05:02:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.