GENE-FL: Gene-Driven Parameter-Efficient Dynamic Federated Learning

• URL: http://arxiv.org/abs/2504.14628v1
• Date: Sun, 20 Apr 2025 14:10:02 GMT
• Title: GENE-FL: Gene-Driven Parameter-Efficient Dynamic Federated Learning
• Authors: Shunxin Guo, Jiaqi Lv, Qiufeng Wang, Xin Geng,
• Abstract summary: We propose a Gene-driven parameter-efficient dynamic Federated Learning (GENE-FL) framework.<n>First, local models perform quadratic constraints based on parameters with high Fisher values in the global model.<n>Second, we apply the strategy of parameter sensitivity analysis in local model parameters to condense the textitlearnGene for interaction.<n>Third, the server aggregates these small-scale trained textitlearnGenes into a robust textitlearnGene with cross-task generalization capability.
• Score: 43.967121817631046
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world \underline{F}ederated \underline{L}earning systems often encounter \underline{D}ynamic clients with \underline{A}gnostic and highly heterogeneous data distributions (DAFL), which pose challenges for efficient communication and model initialization. To address these challenges, we draw inspiration from the recently proposed Learngene paradigm, which compresses the large-scale model into lightweight, cross-task meta-information fragments. Learngene effectively encapsulates and communicates core knowledge, making it particularly well-suited for DAFL, where dynamic client participation requires communication efficiency and rapid adaptation to new data distributions. Based on this insight, we propose a Gene-driven parameter-efficient dynamic Federated Learning (GENE-FL) framework. First, local models perform quadratic constraints based on parameters with high Fisher values in the global model, as these parameters are considered to encapsulate generalizable knowledge. Second, we apply the strategy of parameter sensitivity analysis in local model parameters to condense the \textit{learnGene} for interaction. Finally, the server aggregates these small-scale trained \textit{learnGene}s into a robust \textit{learnGene} with cross-task generalization capability, facilitating the rapid initialization of dynamic agnostic client models. Extensive experimental results demonstrate that GENE-FL reduces \textbf{4 $\times$} communication costs compared to FEDAVG and effectively initializes agnostic client models with only about \textbf{9.04} MB.

Related papers

• Boosting Generalization Performance in Model-Heterogeneous Federated Learning Using Variational Transposed Convolution [0.27309692684728615]
  Federated learning (FL) is a pioneering machine learning paradigm that enables distributed clients to process local data effectively.<n>Traditional model-homogeneous approaches mainly involve debiasing the local training procedures with regularization or dynamically adjusting client weights in aggregation.<n>We propose a model-heterogeneous FL framework that can improve clients' generalization performance over unseen data without model aggregation.
  arXiv  Detail & Related papers  (2025-08-03T08:55:18Z)
• Hypernetworks for Model-Heterogeneous Personalized Federated Learning [13.408669475480824]
  We propose a server-side hypernetwork that takes client-specific embedding vectors as input and outputs personalized parameters tailored to each client's heterogeneous model.<n>To promote knowledge sharing and reduce computation, we introduce a multi-head structure within the hypernetwork, allowing clients with similar model sizes to share heads.<n>Our framework does not rely on external datasets and does not require disclosure of client model architectures.
  arXiv  Detail & Related papers  (2025-07-30T02:24:26Z)
• High-Fidelity Scientific Simulation Surrogates via Adaptive Implicit Neural Representations [51.90920900332569]
  Implicit neural representations (INRs) offer a compact and continuous framework for modeling spatially structured data.<n>Recent approaches address this by introducing additional features along rigid geometric structures.<n>We propose a simple yet effective alternative: Feature-Adaptive INR (FA-INR)
  arXiv  Detail & Related papers  (2025-06-07T16:45:17Z)
• Federated Prototype Graph Learning [33.38948169766356]
  Federated Graph Learning (FGL) has gained significant attention for its distributed training capabilities.<n>FEMAIL: We propose FedPG as a general prototype-guided optimization method for the above multi-level FGL heterogeneity.<n> Experiments demonstrate that FedPG outperforms SOTA baselines by an average of 3.57% in accuracy while reducing communication costs by 168x.
  arXiv  Detail & Related papers  (2025-04-13T09:21:21Z)
• Instruction-Guided Autoregressive Neural Network Parameter Generation [49.800239140036496]
  We propose IGPG, an autoregressive framework that unifies parameter synthesis across diverse tasks and architectures.<n>By autoregressively generating neural network weights' tokens, IGPG ensures inter-layer coherence and enables efficient adaptation across models and datasets.<n>Experiments on multiple datasets demonstrate that IGPG consolidates diverse pretrained models into a single, flexible generative framework.
  arXiv  Detail & Related papers  (2025-04-02T05:50:19Z)
• Data-centric Federated Graph Learning with Large Language Models [34.224475952206404]
  In federated graph learning (FGL), a complete graph is divided into multiple subgraphs stored in each client due to privacy concerns.<n>A pain point of FGL is the heterogeneity problem, where nodes or structures present non-IID properties among clients.<n>We propose a general framework that innovatively decomposes the task of large language models for FGL into two sub-tasks theoretically.
  arXiv  Detail & Related papers  (2025-03-25T08:43:08Z)
• Beyond the Federation: Topology-aware Federated Learning for Generalization to Unseen Clients [10.397502254316645]
  Federated learning is widely employed to tackle distributed sensitive data. Topology-aware Federated Learning (TFL) trains robust models against out-of-federation (OOF) data. We formulate a novel optimization problem for TFL, consisting of two key modules: Client Topology Learning and Learning on Client Topology. Empirical evaluation on a variety of real-world datasets verifies TFL's superior OOF robustness and scalability.
  arXiv  Detail & Related papers  (2024-07-06T03:57:05Z)
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning [18.916282151435727]
  Heterogeneous Federated Learning (HtFL) has attracted attention due to its ability to support heterogeneous models and data. We introduce a novel HtFL approach called FedTGP, which leverages our Adaptive-margin-enhanced Contrastive Learning (ACL) to learn Trainable Global Prototypes (TGP) on the server.
  arXiv  Detail & Related papers  (2024-01-06T14:43:47Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Federated Learning with Neural Graphical Models [2.2721854258621064]
  Federated Learning (FL) addresses the need to create models based on proprietary data. We develop a FL framework which maintains a global NGM model that learns the averaged information from the local NGM models. We experimentally demonstrated the use of FedNGMs for extracting insights from CDC's Infant Mortality dataset.
  arXiv  Detail & Related papers  (2023-09-20T23:24:22Z)
• Adapter-based Selective Knowledge Distillation for Federated Multi-domain Meeting Summarization [36.916155654985936]
  Meeting summarization has emerged as a promising technique for providing users with condensed summaries. We propose adapter-based Federated Selective Knowledge Distillation (AdaFedSelecKD) for training performant client models.
  arXiv  Detail & Related papers  (2023-08-07T03:34:01Z)
• Personalizing Federated Learning with Over-the-Air Computations [84.8089761800994]
  Federated edge learning is a promising technology to deploy intelligence at the edge of wireless networks in a privacy-preserving manner. Under such a setting, multiple clients collaboratively train a global generic model under the coordination of an edge server. This paper presents a distributed training paradigm that employs analog over-the-air computation to address the communication bottleneck.
  arXiv  Detail & Related papers  (2023-02-24T08:41:19Z)
• Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning [86.59588262014456]
  Federated Learning (FL) is an emerging distributed learning paradigm under privacy constraint. We propose a data-free knowledge distillation method to fine-tune the global model in the server (FedFTG) Our FedFTG significantly outperforms the state-of-the-art (SOTA) FL algorithms and can serve as a strong plugin for enhancing FedAvg, FedProx, FedDyn, and SCAFFOLD.
  arXiv  Detail & Related papers  (2022-03-17T11:18:17Z)
• Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generation [56.73834525802723]
  Lightweight Dynamic Graph Convolutional Networks (LDGCNs) are proposed. LDGCNs capture richer non-local interactions by synthesizing higher order information from the input graphs. We develop two novel parameter saving strategies based on the group graph convolutions and weight tied convolutions to reduce memory usage and model complexity.
  arXiv  Detail & Related papers  (2020-10-09T06:03:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.