MePo++: Unifying Representation Refinement and Reconciliation for General Continual Learning

• URL: http://arxiv.org/abs/2609.05075v1
• Date: Fri, 04 Sep 2026 12:33:18 GMT
• Title: MePo++: Unifying Representation Refinement and Reconciliation for General Continual Learning
• Abstract summary: General continual learning (GCL) aims to learn from evolving data streams without task identities, explicit boundaries, or repeated access to previous data.<n>MePo++ bridges pretrained knowledge and downstream GCL through representation refinement and reconciliation.
• Score: 47.304673929751935
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: General continual learning (GCL) aims to learn from evolving data streams without task identities, explicit boundaries, or repeated access to previous data, making it a realistic yet challenging setting for continual intelligence. Although pretrained models (PTMs) provide rich prior knowledge for addressing the limited supervision and non-stationary nature of GCL, existing PTM-based methods often directly adapt pretrained representations and overlook two critical gaps: the misalignment between upstream pretraining and downstream continual adaptation, and the unreliability of conventional output alignment under blurry streams. Here we propose MePo++, a unified post-training framework that bridges pretrained knowledge and downstream GCL through representation refinement and reconciliation. MePo++ introduces two complementary components: MetaPrep, which improves representation plasticity for continual adaptation through unsupervised meta-refinement over pseudo continual sequences; and StreamAlign, which reinforces representation stability by reconciling evolving online features with a stable pretrained geometry. By improving representation learnability before adaptation and preserving alignment during continual learning, MePo++ enables PTMs to remain both plastic for new concepts and stable over evolving streams. Experiments across diverse PTMs, datasets, and continual learning baselines demonstrate the consistent effectiveness and generality of MePo++ for PTM-based GCL. Our code is available at https://github.com/SunGL001/MePo_Plus.

Related papers

• MePo: Meta Post-Refinement for Rehearsal-Free General Continual Learning [47.195830952416294]
  We introduce an innovative approach named Meta Post-Refinement (MePo) for PTMs-based general continual learning (GCL)<n>MePo constructs pseudo task sequences from pretraining data and develops a bi-level meta-learning paradigm to refine the pretrained backbone.<n>MePo serves as a plug-in strategy that achieves significant performance gains across a variety of GCL benchmarks and pretrained checkpoints.
  arXiv  Detail & Related papers  (2026-02-08T12:15:35Z)
• Unbiased Online Curvature Approximation for Regularized Graph Continual Learning [9.70311578832594]
  Graph continual learning (GCL) aims to learn from a continuous sequence of graph-based tasks.<n>Regularization methods are vital for preventing catastrophic forgetting in GCL.<n>We propose a new unbiased online curvature approximation of the full Fisher information matrix (FIM) based on the model's current learning state.
  arXiv  Detail & Related papers  (2025-09-16T06:35:13Z)
• Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgetting [70.83781268763215]
  Vision-language models (VLMs) have achieved impressive performance across diverse multimodal tasks by leveraging large-scale pre-training.<n>VLMs face unique challenges such as cross-modal feature drift, parameter interference due to shared architectures, and zero-shot capability erosion.<n>This survey aims to serve as a comprehensive and diagnostic reference for researchers developing lifelong vision-language systems.
  arXiv  Detail & Related papers  (2025-08-06T09:03:10Z)
• Adapt before Continual Learning [9.477667054965782]
  Adapting PTMs before the core CL process (ACL) is a novel framework that introduces a plug-and-play adaptation phase prior to learning each new task.<n>ACL significantly improves CL performance across benchmarks and integrated methods.
  arXiv  Detail & Related papers  (2025-06-04T13:46:33Z)
• Continual Task Learning through Adaptive Policy Self-Composition [54.95680427960524]
  CompoFormer is a structure-based continual transformer model that adaptively composes previous policies via a meta-policy network. Our experiments reveal that CompoFormer outperforms conventional continual learning (CL) methods, particularly in longer task sequences.
  arXiv  Detail & Related papers  (2024-11-18T08:20:21Z)
• Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization [77.62516752323207]
  We introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization. A self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario.
  arXiv  Detail & Related papers  (2024-07-11T10:35:53Z)
• Transformers for Supervised Online Continual Learning [11.270594318662233]
  We propose a method that leverages transformers' in-context learning capabilities for online continual learning. Our method demonstrates significant improvements over previous state-of-the-art results on CLOC, a challenging large-scale real-world benchmark for image geo-localization.
  arXiv  Detail & Related papers  (2024-03-03T16:12:20Z)
• CTP: Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology Preservation [128.00940554196976]
  Vision-Language Continual Pretraining (VLCP) has shown impressive results on diverse downstream tasks by offline training on large-scale datasets. To support the study of Vision-Language Continual Pretraining (VLCP), we first contribute a comprehensive and unified benchmark dataset P9D. The data from each industry as an independent task supports continual learning and conforms to the real-world long-tail nature to simulate pretraining on web data.
  arXiv  Detail & Related papers  (2023-08-14T13:53:18Z)
• Recyclable Tuning for Continual Pre-training [98.51583779792031]
  Continual pre-training is the paradigm where pre-trained language models (PLMs) continually acquire fresh knowledge from growing data and gradually get upgraded. We contend that proper algorithms for recycling outdated adapted weights should be developed. We show that both methods can be combined to achieve better performance.
  arXiv  Detail & Related papers  (2023-05-15T15:05:44Z)
• New Insights for the Stability-Plasticity Dilemma in Online Continual Learning [21.664470275289407]
  We propose an online continual learning framework named multi-scale feature adaptation network (MuFAN) MuFAN outperforms other state-of-the-art continual learning methods on the SVHN, CIFAR100, miniImageNet, and CORe50 datasets.
  arXiv  Detail & Related papers  (2023-02-17T07:43:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.