Resource-Adaptive Federated Text Generation with Differential Privacy

• URL: http://arxiv.org/abs/2603.07027v1
• Date: Sat, 07 Mar 2026 04:11:58 GMT
• Title: Resource-Adaptive Federated Text Generation with Differential Privacy
• Abstract summary: differentially private (DP) synthetic datasets approximate the global distribution and can be reused across tasks.<n>We propose a flexible participation framework that adapts to client capacities.
• Score: 3.7397653111851814
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In cross-silo federated learning (FL), sensitive text datasets remain confined to local organizations due to privacy regulations, making repeated training for each downstream task both communication-intensive and privacy-demanding. A promising alternative is to generate differentially private (DP) synthetic datasets that approximate the global distribution and can be reused across tasks. However, pretrained large language models (LLMs) often fail under domain shift, and federated finetuning is hindered by computational heterogeneity: only resource-rich clients can update the model, while weaker clients are excluded, amplifying data skew and the adverse effects of DP noise. We propose a flexible participation framework that adapts to client capacities. Strong clients perform DP federated finetuning, while weak clients contribute through a lightweight DP voting mechanism that refines synthetic text. To ensure the synthetic data mirrors the global dataset, we apply control codes (e.g., labels, topics, metadata) that represent each client's data proportions and constrain voting to semantically coherent subsets. This two-phase approach requires only a single round of communication for weak clients and integrates contributions from all participants. Experiments show that our framework improves distribution alignment and downstream robustness under DP and heterogeneity.

Related papers

• Separate Aggregation of Split Network for Personalized Federated Learning [0.9668407688201359]
  Federated learning can degrade substantially under heterogeneous client data distributions.<n>We propose PGFedSplit, a personalized federated learning framework.<n>Experiments on Fashion MNIST, CIFAR 10, CIFAR 100, and Tiny ImageNet demonstrate consistent improvements over state of the art PFL methods.
  arXiv  Detail & Related papers  (2026-05-26T05:44:30Z)
• Concordia: Self-Improving Synthetic Tables for Federated LLMs [80.03837595689608]
  Federated learning enables training large language models (LLMs) without sharing raw data.<n>We propose Concordia, a framework that aligns synthetic data generation with federated validation utility.
  arXiv  Detail & Related papers  (2026-05-11T01:17:58Z)
• Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketching [19.959825317183924]
  Federated Learning (FL) enables collaborative training across decentralized data.<n>We propose pFed1BS, a novel personalized FL framework that achieves extreme communication compression through one-bit random sketching.<n>In our framework, clients transmit highly compressed one-bit sketches, and the server aggregates and broadcasts a global one-bit consensus.
  arXiv  Detail & Related papers  (2025-11-17T08:55:22Z)
• Towards Federated Clustering: A Client-wise Private Graph Aggregation Framework [57.04850867402913]
  Federated clustering addresses the challenge of extracting patterns from decentralized, unlabeled data.<n>We propose Structural Privacy-Preserving Federated Graph Clustering (SPP-FGC), a novel algorithm that innovatively leverages local structural graphs as the primary medium for privacy-preserving knowledge sharing.<n>Our framework achieves state-of-the-art performance, improving clustering accuracy by up to 10% (NMI) over federated baselines while maintaining provable privacy guarantees.
  arXiv  Detail & Related papers  (2025-11-14T03:05:22Z)
• CO-PFL: Contribution-Oriented Personalized Federated Learning for Heterogeneous Networks [51.43780477302533]
  Contribution-Oriented PFL (CO-PFL) is a novel algorithm that dynamically estimates each client's contribution for global aggregation.<n>CO-PFL consistently surpasses state-of-the-art methods in robustness in personalization accuracy, robustness, scalability and convergence stability.
  arXiv  Detail & Related papers  (2025-10-23T05:10:06Z)
• FedPPA: Progressive Parameter Alignment for Personalized Federated Learning [0.9931624906346306]
  Federated Learning (FL) is designed as a decentralized, privacy-preserving machine learning paradigm that enables multiple clients to collaboratively train a model without sharing their data.<n>In real-world scenarios, clients often have heterogeneous computational resources and hold non-independent and identically distributed data (non-IID), which poses significant challenges during training.<n>We propose Progressive Alignment (FedPPA), which progressively aligns the weights of common layers across clients with the global model's weights.<n>Experiments on three image classification datasets, including MNIST, FMNIST, and CIFAR-10, demonstrate that FedPPA consistently outperforms existing FL algorithms,
  arXiv  Detail & Related papers  (2025-10-16T14:03:05Z)
• FedHFT: Efficient Federated Finetuning with Heterogeneous Edge Clients [37.60974510385015]
  Fine-tuning pre-trained large language models (LLMs) has become a common practice for personalized natural language understanding (NLU) applications.<n>There are two main challenges: (i) limited and/or heterogeneous data for fine-tuning due to proprietary data confidentiality or privacy requirements, and (ii) varying resources available across participating clients such as edge devices.<n>This paper presents FedHFT - an efficient and personalized federated fine-tuning framework to address both challenges.
  arXiv  Detail & Related papers  (2025-10-15T19:47:51Z)
• Hybrid-Regularized Magnitude Pruning for Robust Federated Learning under Covariate Shift [2.298932494750101]
  We show that inconsistencies in client-side training distributions substantially degrade the performance of federated learning models.<n>We propose a novel FL framework using a combination of pruning and regularisation of clients' training to improve the sparsity, redundancy, and robustness of neural connections.
  arXiv  Detail & Related papers  (2024-12-19T16:22:37Z)
• Communication-Efficient Personalized Federated Learning for Speech-to-Text Tasks [64.02867484165476]
  To protect privacy and meet legal regulations, federated learning (FL) has gained significant attention for training speech-to-text (S2T) systems.<n>The commonly used FL approach (i.e., textscFedAvg) in S2T tasks typically suffers from extensive communication overhead.<n>We propose a personalized federated S2T framework that introduces textscFedLoRA, a lightweight LoRA module for client-side tuning and interaction with the server, and textscFedMem, a global model equipped with a $k$-near
  arXiv  Detail & Related papers  (2024-01-18T15:39:38Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• PS-FedGAN: An Efficient Federated Learning Framework Based on Partially Shared Generative Adversarial Networks For Data Privacy [56.347786940414935]
  Federated Learning (FL) has emerged as an effective learning paradigm for distributed computation. This work proposes a novel FL framework that requires only partial GAN model sharing. Named as PS-FedGAN, this new framework enhances the GAN releasing and training mechanism to address heterogeneous data distributions.
  arXiv  Detail & Related papers  (2023-05-19T05:39:40Z)
• Federated Learning with GAN-based Data Synthesis for Non-IID Clients [8.304185807036783]
  Federated learning (FL) has recently emerged as a popular privacy-preserving collaborative learning paradigm. We propose a novel framework, named Synthetic Data Aided Federated Learning (SDA-FL), to resolve this non-IID challenge by sharing synthetic data.
  arXiv  Detail & Related papers  (2022-06-11T11:43:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.