MePo: Meta Post-Refinement for Rehearsal-Free General Continual Learning

• URL: http://arxiv.org/abs/2602.07940v2
• Date: Wed, 11 Feb 2026 09:48:31 GMT
• Title: MePo: Meta Post-Refinement for Rehearsal-Free General Continual Learning
• Authors: Guanglong Sun, Hongwei Yan, Liyuan Wang, Zhiqi Kang, Shuang Cui, Hang Su, Jun Zhu, Yi Zhong,
• Abstract summary: We introduce an innovative approach named Meta Post-Refinement (MePo) for PTMs-based general continual learning (GCL)<n>MePo constructs pseudo task sequences from pretraining data and develops a bi-level meta-learning paradigm to refine the pretrained backbone.<n>MePo serves as a plug-in strategy that achieves significant performance gains across a variety of GCL benchmarks and pretrained checkpoints.
• Score: 47.195830952416294
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To cope with uncertain changes of the external world, intelligent systems must continually learn from complex, evolving environments and respond in real time. This ability, collectively known as general continual learning (GCL), encapsulates practical challenges such as online datastreams and blurry task boundaries. Although leveraging pretrained models (PTMs) has greatly advanced conventional continual learning (CL), these methods remain limited in reconciling the diverse and temporally mixed information along a single pass, resulting in sub-optimal GCL performance. Inspired by meta-plasticity and reconstructive memory in neuroscience, we introduce here an innovative approach named Meta Post-Refinement (MePo) for PTMs-based GCL. This approach constructs pseudo task sequences from pretraining data and develops a bi-level meta-learning paradigm to refine the pretrained backbone, which serves as a prolonged pretraining phase but greatly facilitates rapid adaptation of representation learning to downstream GCL tasks. MePo further initializes a meta covariance matrix as the reference geometry of pretrained representation space, enabling GCL to exploit second-order statistics for robust output alignment. MePo serves as a plug-in strategy that achieves significant performance gains across a variety of GCL benchmarks and pretrained checkpoints in a rehearsal-free manner (e.g., 15.10\%, 13.36\%, and 12.56\% on CIFAR-100, ImageNet-R, and CUB-200 under Sup-21/1K). Our source code is available at \href{https://github.com/SunGL001/MePo}{MePo}

Related papers

• FlyPrompt: Brain-Inspired Random-Expanded Routing with Temporal-Ensemble Experts for General Continual Learning [28.78785349011036]
  General continual learning (GCL) challenges intelligent systems to learn from single-pass, non-stationary data streams without clear task boundaries.<n>FlyPrompt is a brain-inspired framework that decomposes GCL into two subproblems: expert routing and expert competence improvement.
  arXiv  Detail & Related papers  (2026-02-02T11:32:56Z)
• Preserving Cross-Modal Consistency for CLIP-based Class-Incremental Learning [77.82901519692378]
  Class-incremental learning (CIL) enables models to continuously learn new categories from sequential tasks without forgetting previously acquired knowledge.<n>We propose DMC, a two-stage framework for CLIP-based CIL that decouples the adaptation of the vision encoder and the optimization of textual soft prompts.<n>Extensive experiments on CIFAR-100, Imagenet-R, CUB-200, and UCF-101 demonstrate that both DMC and DMC-OT achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-11-14T05:36:36Z)
• Unbiased Online Curvature Approximation for Regularized Graph Continual Learning [9.70311578832594]
  Graph continual learning (GCL) aims to learn from a continuous sequence of graph-based tasks.<n>Regularization methods are vital for preventing catastrophic forgetting in GCL.<n>We propose a new unbiased online curvature approximation of the full Fisher information matrix (FIM) based on the model's current learning state.
  arXiv  Detail & Related papers  (2025-09-16T06:35:13Z)
• Deepfake Detection that Generalizes Across Benchmarks [48.85953407706351]
  The generalization of deepfake detectors to unseen manipulation techniques remains a challenge for practical deployment.<n>This work demonstrates that robust generalization is achievable through a parameter-efficient adaptation of one of the foundational pre-trained vision encoders.<n>The proposed method achieves state-of-the-art performance, outperforming more complex, recent approaches in average cross-dataset AUROC.
  arXiv  Detail & Related papers  (2025-08-08T12:03:56Z)
• Towards Improving Long-Tail Entity Predictions in Temporal Knowledge Graphs through Global Similarity and Weighted Sampling [53.11315884128402]
  Temporal Knowledge Graph (TKG) completion models traditionally assume access to the entire graph during training.<n>We present an incremental training framework specifically designed for TKGs, aiming to address entities that are either not observed during training or have sparse connections.<n>Our approach combines a model-agnostic enhancement layer with a weighted sampling strategy, that can be augmented to and improve any existing TKG completion method.
  arXiv  Detail & Related papers  (2025-07-25T06:02:48Z)
• MEGA: Second-Order Gradient Alignment for Catastrophic Forgetting Mitigation in GFSCIL [9.557104125817668]
  Graph Few-Shot Class-Incremental Learning (GFSCIL) enables models to continually learn from limited samples of novel tasks after initial training on a large base dataset.<n>Existing GFSCIL approaches typically utilize Prototypical Networks (PNs) for metric-based class representations and fine-tune the model during the incremental learning stage.<n>We introduce Model-Agnostic Meta Graph Continual Learning (MEGA), aimed at effectively alleviating catastrophic forgetting for GFSCIL.
  arXiv  Detail & Related papers  (2025-04-18T13:48:15Z)
• Advancing Prompt-Based Methods for Replay-Independent General Continual Learning [44.94466949172424]
  General continual learning (GCL) is a broad concept to describe real-world continual learning (CL) problems.<n>Such requirements result in poor initial performance, limited generalizability, and severe catastrophic forgetting.<n>We propose an innovative approach named MISA (Mask and Initial Session Adaption) to advance prompt-based methods in GCL.
  arXiv  Detail & Related papers  (2025-03-02T00:58:18Z)
• CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning [17.614980614656407]
  We propose Continual Generative training for Incremental prompt-Learning. We exploit Variational Autoencoders to learn class-conditioned distributions. We show that such a generative replay approach can adapt to new tasks while improving zero-shot capabilities.
  arXiv  Detail & Related papers  (2024-07-22T16:51:28Z)
• A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation [121.0693322732454]
  Contrastive Language-Image Pretraining (CLIP) has gained popularity for its remarkable zero-shot capacity. Recent research has focused on developing efficient fine-tuning methods to enhance CLIP's performance in downstream tasks. We revisit a classical algorithm, Gaussian Discriminant Analysis (GDA), and apply it to the downstream classification of CLIP.
  arXiv  Detail & Related papers  (2024-02-06T15:45:27Z)
• Learning towards Synchronous Network Memorizability and Generalizability for Continual Segmentation across Multiple Sites [52.84959869494459]
  In clinical practice, a segmentation network is often required to continually learn on a sequential data stream from multiple sites. Existing methods are usually restricted in either network memorizability on previous sites or generalizability on unseen sites. This paper aims to tackle the problem of Synchronous Memorizability and Generalizability with a novel proposed SMG-learning framework.
  arXiv  Detail & Related papers  (2022-06-14T13:04:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.