Improving Generalization in Heterogeneous Federated Continual Learning via Spatio-Temporal Gradient Matching with Prototypical Coreset

• URL: http://arxiv.org/abs/2506.12031v1
• Date: Thu, 22 May 2025 18:26:51 GMT
• Title: Improving Generalization in Heterogeneous Federated Continual Learning via Spatio-Temporal Gradient Matching with Prototypical Coreset
• Authors: Minh-Duong Nguyen, Le-Tuan Nguyen, Quoc-Viet Pham,
• Abstract summary: This paper explores a more practical and challenging Federated Continual Learning setting, where clients may have unrelated or even conflicting data and tasks.<n>Existing FCL approaches often use generative replay to create pseudo-datasets of previous tasks.<n>To address these challenges, we propose a novel approach called Spatio-Temporal grAdient Matching with network-free Prototype (STAMP)
• Score: 4.1751318268724384
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Continual Learning (FCL) has recently emerged as a crucial research area, as data from distributed clients typically arrives as a stream, requiring sequential learning. This paper explores a more practical and challenging FCL setting, where clients may have unrelated or even conflicting data and tasks. In this scenario, statistical heterogeneity and data noise can create spurious correlations, leading to biased feature learning and catastrophic forgetting. Existing FCL approaches often use generative replay to create pseudo-datasets of previous tasks. However, generative replay itself suffers from catastrophic forgetting and task divergence among clients, leading to overfitting in FCL. Existing FCL approaches often use generative replay to create pseudo-datasets of previous tasks. However, generative replay itself suffers from catastrophic forgetting and task divergence among clients, leading to overfitting in FCL. To address these challenges, we propose a novel approach called Spatio-Temporal grAdient Matching with network-free Prototype (STAMP). Our contributions are threefold: 1) We develop a model-agnostic method to determine subset of samples that effectively form prototypes when using a prototypical network, making it resilient to continual learning challenges; 2) We introduce a spatio-temporal gradient matching approach, applied at both the client-side (temporal) and server-side (spatial), to mitigate catastrophic forgetting and data heterogeneity; 3) We leverage prototypes to approximate task-wise gradients, improving gradient matching on the client-side. Extensive experiments demonstrate our method's superiority over existing baselines.

Related papers

• Accurate Forgetting for Heterogeneous Federated Continual Learning [89.08735771893608]
  We propose a new concept accurate forgetting (AF) and develop a novel generative-replay methodMethodwhich selectively utilizes previous knowledge in federated networks.<n>We employ a probabilistic framework based on a normalizing flow model to quantify the credibility of previous knowledge.
  arXiv  Detail & Related papers  (2025-02-20T02:35:17Z)
• Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learning [13.836798036474143]
  Key challenge in Federated Class Continual Learning is catastrophic forgetting. We propose a novel method of data replay based on diffusion models. Our method significantly outperforms existing baselines.
  arXiv  Detail & Related papers  (2024-09-02T10:07:24Z)
• Reshaping the Online Data Buffering and Organizing Mechanism for Continual Test-Time Adaptation [49.53202761595912]
  Continual Test-Time Adaptation involves adapting a pre-trained source model to continually changing unsupervised target domains. We analyze the challenges of this task: online environment, unsupervised nature, and the risks of error accumulation and catastrophic forgetting. We propose an uncertainty-aware buffering approach to identify and aggregate significant samples with high certainty from the unsupervised, single-pass data stream.
  arXiv  Detail & Related papers  (2024-07-12T15:48:40Z)
• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• Finding Foundation Models for Time Series Classification with a PreText Task [7.197233473373693]
  This paper introduces pre-trained domain foundation models for Time Series Classification. A key aspect of our methodology is a novel pretext task that spans multiple datasets. Our experiments on the UCR archive demonstrate that this pre-training strategy significantly outperforms the conventional training approach without pre-training.
  arXiv  Detail & Related papers  (2023-11-24T15:03:55Z)
• Time-series Generation by Contrastive Imitation [87.51882102248395]
  We study a generative framework that seeks to combine the strengths of both: Motivated by a moment-matching objective to mitigate compounding error, we optimize a local (but forward-looking) transition policy. At inference, the learned policy serves as the generator for iterative sampling, and the learned energy serves as a trajectory-level measure for evaluating sample quality.
  arXiv  Detail & Related papers  (2023-11-02T16:45:25Z)
• Prior-Free Continual Learning with Unlabeled Data in the Wild [24.14279172551939]
  We propose a Prior-Free Continual Learning (PFCL) method to incrementally update a trained model on new tasks. PFCL learns new tasks without knowing the task identity or any previous data. Our experiments show that our PFCL method significantly mitigates forgetting in all three learning scenarios.
  arXiv  Detail & Related papers  (2023-10-16T13:59:56Z)
• FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Data [8.677832361022809]
  Federated learning allows multiple clients to collaboratively train a model without exchanging their data. Common solutions involve an auxiliary loss to regularize weight divergence or feature inconsistency during local training. We propose a novel framework named Federated learning with Feature Anchors (FedFA)
  arXiv  Detail & Related papers  (2022-11-17T02:27:44Z)
• Contrastive Self-supervised Sequential Recommendation with Robust Augmentation [101.25762166231904]
  Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data. Old and new issues remain, including data-sparsity and noisy data. We propose Contrastive Self-Supervised Learning for sequential Recommendation (CoSeRec)
  arXiv  Detail & Related papers  (2021-08-14T07:15:25Z)
• Recent Developments Combining Ensemble Smoother and Deep Generative Networks for Facies History Matching [58.720142291102135]
  This research project focuses on the use of autoencoders networks to construct a continuous parameterization for facies models. We benchmark seven different formulations, including VAE, generative adversarial network (GAN), Wasserstein GAN, variational auto-encoding GAN, principal component analysis (PCA) with cycle GAN, PCA with transfer style network and VAE with style loss.
  arXiv  Detail & Related papers  (2020-05-08T21:32:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.