Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projection

• URL: http://arxiv.org/abs/2509.21606v1
• Date: Thu, 25 Sep 2025 21:20:00 GMT
• Title: Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projection
• Authors: Seohyeon Cha, Huancheng Chen, Haris Vikalo,
• Abstract summary: Federated continual learning (FCL) enables distributed client devices to learn from streaming data across diverse and evolving tasks.<n>We propose Federated gradient Projection-based Continual Learning with Task Identity Prediction (FedProTIP)<n>This framework mitigates forgetting by projecting client updates onto the complement of the subspace spanned by previously learned representations of the global model.
• Score: 23.99793728516052
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Federated continual learning (FCL) enables distributed client devices to learn from streaming data across diverse and evolving tasks. A major challenge to continual learning, catastrophic forgetting, is exacerbated in decentralized settings by the data heterogeneity, constrained communication and privacy concerns. We propose Federated gradient Projection-based Continual Learning with Task Identity Prediction (FedProTIP), a novel FCL framework that mitigates forgetting by projecting client updates onto the orthogonal complement of the subspace spanned by previously learned representations of the global model. This projection reduces interference with earlier tasks and preserves performance across the task sequence. To further address the challenge of task-agnostic inference, we incorporate a lightweight mechanism that leverages core bases from prior tasks to predict task identity and dynamically adjust the global model's outputs. Extensive experiments across standard FCL benchmarks demonstrate that FedProTIP significantly outperforms state-of-the-art methods in average accuracy, particularly in settings where task identities are a priori unknown.

Related papers

• Rethinking Federated Graph Foundation Models: A Graph-Language Alignment-based Approach [8.517604507672262]
  Recent studies of federated graph foundational models (FedGFMs) break the idealized and untenable assumption of having centralized data storage to train graph foundation models.<n>Existing studies that project aligned generalizable knowledge onto a discrete token space via vector-quantized backbones suffer from irreversible knowledge loss during the quantization process.
  arXiv  Detail & Related papers  (2026-01-29T07:50:00Z)
• Forget Less, Retain More: A Lightweight Regularizer for Rehearsal-Based Continual Learning [51.07663354001582]
  Deep neural networks suffer from catastrophic forgetting, where performance on previous tasks degrades after training on a new task.<n>We present a novel approach to address this challenge, focusing on the intersection of memory-based methods and regularization approaches.<n>We formulate a regularization strategy, termed Information Maximization (IM) regularizer, for memory-based continual learning methods.
  arXiv  Detail & Related papers  (2025-12-01T15:56:00Z)
• FedGTEA: Federated Class-Incremental Learning with Gaussian Task Embedding and Alignment [4.031060983228927]
  We introduce a novel framework for Federated Class Incremental Learning, called Federated Gaussian Task Embedding and Alignment (FedGTEA)<n>FedGTEA is designed to capture task-specific knowledge and model uncertainty in a scalable and communication-efficient manner.
  arXiv  Detail & Related papers  (2025-10-14T19:02:41Z)
• Personalized Vision via Visual In-Context Learning [62.85784251383279]
  We present a visual in-context learning framework for personalized vision.<n>PICO infers the underlying transformation and applies it to new inputs without retraining.<n>We also propose an attention-guided seed scorer that improves reliability via efficient inference scaling.
  arXiv  Detail & Related papers  (2025-09-29T17:58:45Z)
• Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgetting [70.83781268763215]
  Vision-language models (VLMs) have achieved impressive performance across diverse multimodal tasks by leveraging large-scale pre-training.<n>VLMs face unique challenges such as cross-modal feature drift, parameter interference due to shared architectures, and zero-shot capability erosion.<n>This survey aims to serve as a comprehensive and diagnostic reference for researchers developing lifelong vision-language systems.
  arXiv  Detail & Related papers  (2025-08-06T09:03:10Z)
• Orthogonal Projection Subspace to Aggregate Online Prior-knowledge for Continual Test-time Adaptation [67.80294336559574]
  Continual Test Time Adaptation (CTTA) is a task that requires a source pre-trained model to continually adapt to new scenarios.<n>We propose a novel pipeline, Orthogonal Projection Subspace to aggregate online Prior-knowledge, dubbed OoPk.
  arXiv  Detail & Related papers  (2025-06-23T18:17:39Z)
• Improving Generalization in Heterogeneous Federated Continual Learning via Spatio-Temporal Gradient Matching with Prototypical Coreset [4.1751318268724384]
  This paper explores a more practical and challenging Federated Continual Learning setting, where clients may have unrelated or even conflicting data and tasks.<n>Existing FCL approaches often use generative replay to create pseudo-datasets of previous tasks.<n>To address these challenges, we propose a novel approach called Spatio-Temporal grAdient Matching with network-free Prototype (STAMP)
  arXiv  Detail & Related papers  (2025-05-22T18:26:51Z)
• Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning [19.27175827358111]
  Continual learning in large language models (LLMs) is prone to catastrophic forgetting, where adapting to new tasks significantly degrades performance on previously learned ones.<n>We propose a novel continual full fine-tuning approach leveraging adaptive singular value decomposition (SVD)<n>We evaluate our approach extensively on standard continual learning benchmarks using both encoder-decoder (T5-Large) and decoder-only (LLaMA-2 7B) models.
  arXiv  Detail & Related papers  (2025-04-09T17:59:42Z)
• Federated Continual Instruction Tuning [39.344583304181135]
  Federated learning (FL) has the potential to leverage all distributed data and training resources to reduce the overhead of joint training.<n>In this work, we introduce the Federated Continual Instruction Tuning (FCIT) benchmark to model this real-world challenge.
  arXiv  Detail & Related papers  (2025-03-17T07:58:06Z)
• Model Predictive Task Sampling for Efficient and Robust Adaptation [46.92143725900031]
  We introduce Model Predictive Task Sampling (MPTS), a framework that bridges the task space and adaptation risk landscape.<n>MPTS employs a generative model to characterize the episodic optimization process and predicts task-specific adaptation risk via posterior inference.<n>MPTS seamlessly integrates into zero-shot, few-shot, and supervised finetuning settings.
  arXiv  Detail & Related papers  (2025-01-19T13:14:53Z)
• Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization [77.62516752323207]
  We introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization. A self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario.
  arXiv  Detail & Related papers  (2024-07-11T10:35:53Z)
• Personalized Wireless Federated Learning for Large Language Models [75.22457544349668]
  Large language models (LLMs) have driven profound transformations in wireless networks.<n>Within wireless environments, the training of LLMs faces significant challenges related to security and privacy.<n>This paper presents a systematic analysis of the training stages of LLMs in wireless networks, including pre-training, instruction tuning, and alignment tuning.
  arXiv  Detail & Related papers  (2024-04-20T02:30:21Z)
• FedEGG: Federated Learning with Explicit Global Guidance [90.04705121816185]
  Federated Learning (FL) holds great potential for diverse applications owing to its privacy-preserving nature.<n>Existing methods help address these challenges via optimization-based client constraints, adaptive client selection, or the use of pre-trained models or synthetic data.<n>We present bftextFedEGG, a new FL algorithm that constructs a global guiding task using a well-defined, easy-to-converge learning task.
  arXiv  Detail & Related papers  (2024-04-18T04:25:21Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• Task Agnostic Representation Consolidation: a Self-supervised based Continual Learning Approach [14.674494335647841]
  We propose a two-stage training paradigm for CL that intertwines task-agnostic and task-specific learning. We show that our training paradigm can be easily added to memory- or regularization-based approaches.
  arXiv  Detail & Related papers  (2022-07-13T15:16:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.