SacFL: Self-Adaptive Federated Continual Learning for Resource-Constrained End Devices

• URL: http://arxiv.org/abs/2505.00365v1
• Date: Thu, 01 May 2025 07:26:35 GMT
• Title: SacFL: Self-Adaptive Federated Continual Learning for Resource-Constrained End Devices
• Authors: Zhengyi Zhong, Weidong Bao, Ji Wang, Jianguo Chen, Lingjuan Lyu, Wei Yang Bryan Lim,
• Abstract summary: Dynamic nature of data, characterized by continuous changes or data drift, poses significant challenges for on-device machine learning models.<n>Traditional centralized approach to continual learning is unsuitable for end devices due to privacy and data volume concerns.<n>FCL emerges as a promising solution, preserving user data locally while enhancing models through collaborative updates.
• Score: 38.27552394580526
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The proliferation of end devices has led to a distributed computing paradigm, wherein on-device machine learning models continuously process diverse data generated by these devices. The dynamic nature of this data, characterized by continuous changes or data drift, poses significant challenges for on-device models. To address this issue, continual learning (CL) is proposed, enabling machine learning models to incrementally update their knowledge and mitigate catastrophic forgetting. However, the traditional centralized approach to CL is unsuitable for end devices due to privacy and data volume concerns. In this context, federated continual learning (FCL) emerges as a promising solution, preserving user data locally while enhancing models through collaborative updates. Aiming at the challenges of limited storage resources for CL, poor autonomy in task shift detection, and difficulty in coping with new adversarial tasks in FCL scenario, we propose a novel FCL framework named SacFL. SacFL employs an Encoder-Decoder architecture to separate task-robust and task-sensitive components, significantly reducing storage demands by retaining lightweight task-sensitive components for resource-constrained end devices. Moreover, $\rm{SacFL}$ leverages contrastive learning to introduce an autonomous data shift detection mechanism, enabling it to discern whether a new task has emerged and whether it is a benign task. This capability ultimately allows the device to autonomously trigger CL or attack defense strategy without additional information, which is more practical for end devices. Comprehensive experiments conducted on multiple text and image datasets, such as Cifar100 and THUCNews, have validated the effectiveness of $\rm{SacFL}$ in both class-incremental and domain-incremental scenarios. Furthermore, a demo system has been developed to verify its practicality.

Related papers

• Using Diffusion Models as Generative Replay in Continual Federated Learning -- What will Happen? [28.465434174993007]
  This paper introduces DCFL, a novel framework tailored to address the challenges of continuous learning in distributed learning environments. Our approach harnesses the power of the conditional diffusion model to generate synthetic historical data at each local device during communication.
  arXiv  Detail & Related papers  (2024-11-10T22:48:21Z)
• Digital Twin-Assisted Federated Learning with Blockchain in Multi-tier Computing Systems [67.14406100332671]
  In Industry 4.0 systems, resource-constrained edge devices engage in frequent data interactions. This paper proposes a digital twin (DT) and federated digital twin (FL) scheme. The efficacy of our proposed cooperative interference-based FL process has been verified through numerical analysis.
  arXiv  Detail & Related papers  (2024-11-04T17:48:02Z)
• INTELLECT: Adapting Cyber Threat Detection to Heterogeneous Computing Environments [0.055923945039144884]
  This paper introduces INTELLECT, a novel solution that integrates feature selection, model pruning, and fine-tuning techniques into a cohesive pipeline for the dynamic adaptation of pre-trained ML models and configurations for IDSs. We demonstrate the advantages of incorporating knowledge distillation techniques while fine-tuning, enabling the ML model to consistently adapt to local network patterns while preserving historical knowledge.
  arXiv  Detail & Related papers  (2024-07-17T22:34:29Z)
• CRSFL: Cluster-based Resource-aware Split Federated Learning for Continuous Authentication [5.636155173401658]
  Split Learning (SL) and Federated Learning (FL) have emerged as promising technologies for training a decentralized Machine Learning (ML) model. We propose combining these technologies to address the continuous authentication challenge while protecting user privacy.
  arXiv  Detail & Related papers  (2024-05-12T06:08:21Z)
• Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters [65.15700861265432]
  We present a parameter-efficient continual learning framework to alleviate long-term forgetting in incremental learning with vision-language models. Our approach involves the dynamic expansion of a pre-trained CLIP model, through the integration of Mixture-of-Experts (MoE) adapters. To preserve the zero-shot recognition capability of vision-language models, we introduce a Distribution Discriminative Auto-Selector.
  arXiv  Detail & Related papers  (2024-03-18T08:00:23Z)
• Task-Distributionally Robust Data-Free Meta-Learning [99.56612787882334]
  Data-Free Meta-Learning (DFML) aims to efficiently learn new tasks by leveraging multiple pre-trained models without requiring their original training data. For the first time, we reveal two major challenges hindering their practical deployments: Task-Distribution Shift ( TDS) and Task-Distribution Corruption (TDC)
  arXiv  Detail & Related papers  (2023-11-23T15:46:54Z)
• Cloud-Device Collaborative Adaptation to Continual Changing Environments in the Real-world [20.547119604004774]
  We propose a new learning paradigm of Cloud-Device Collaborative Continual Adaptation, which encourages collaboration between cloud and device. We also propose an Uncertainty-based Visual Prompt Adapted (U-VPA) teacher-student model to transfer the generalization capability of the large model on the cloud to the device model. Our proposed U-VPA teacher-student framework outperforms previous state-of-the-art test time adaptation and device-cloud collaboration methods.
  arXiv  Detail & Related papers  (2022-12-02T05:02:36Z)
• Online Data Selection for Federated Learning with Limited Storage [53.46789303416799]
  Federated Learning (FL) has been proposed to achieve distributed machine learning among networked devices. The impact of on-device storage on the performance of FL is still not explored. In this work, we take the first step to consider the online data selection for FL with limited on-device storage.
  arXiv  Detail & Related papers  (2022-09-01T03:27:33Z)
• RoFL: Attestable Robustness for Secure Federated Learning [59.63865074749391]
  Federated Learning allows a large number of clients to train a joint model without the need to share their private data. To ensure the confidentiality of the client updates, Federated Learning systems employ secure aggregation. We present RoFL, a secure Federated Learning system that improves robustness against malicious clients.
  arXiv  Detail & Related papers  (2021-07-07T15:42:49Z)
• MDLdroid: a ChainSGD-reduce Approach to Mobile Deep Learning for Personal Mobile Sensing [14.574274428615666]
  Running deep learning on devices offers several advantages including data privacy preservation and low-latency response for both model robustness and update. Personal mobile sensing applications are mostly user-specific and highly affected by environment. We present MDLdroid, a novel decentralized mobile deep learning framework to enable resource-aware on-device collaborative learning.
  arXiv  Detail & Related papers  (2020-02-07T16:55:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.