Federated Instruction Tuning of LLMs with Domain Coverage Augmentation

• URL: http://arxiv.org/abs/2409.20135v5
• Date: Tue, 21 Jan 2025 09:25:25 GMT
• Title: Federated Instruction Tuning of LLMs with Domain Coverage Augmentation
• Authors: Zezhou Wang, Yaxin Du, Xingjun Ma, Yugang Jiang, Zhuzhong Qian, Siheng Chen,
• Abstract summary: Federated Domain-specific Instruction Tuning (FedDIT) utilizes limited cross-client private data together with various strategies of instruction augmentation.<n>We propose FedDCA, which optimize domain coverage through greedy client center selection and retrieval-based augmentation.<n>For client-side computational efficiency and system scalability, FedDCA$*$, the variant of FedDCA, utilizes heterogeneous encoders with server-side feature alignment.
• Score: 87.49293964617128
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated Domain-specific Instruction Tuning (FedDIT) utilizes limited cross-client private data together with various strategies of instruction augmentation, ultimately boosting model performance within specific domains. To date, the factors affecting FedDIT remain unclear, and existing instruction augmentation methods primarily focus on the centralized setting without considering distributed environments. Our experiments reveal that the cross-client domain coverage, rather than data heterogeneity, drives model performance in FedDIT. In response, we propose FedDCA, which optimizes domain coverage through greedy client center selection and retrieval-based augmentation. At its core, the greedy selection procedure iteratively picks client centers that maximize the diversity and coverage of the instruction space while avoiding redundancy with previously selected centers. This ensures broad yet efficient coverage of the domain distribution across clients. For client-side computational efficiency and system scalability, FedDCA$^*$, the variant of FedDCA, utilizes heterogeneous encoders with server-side feature alignment. Extensive experiments across code, medical, financial, and mathematical domains substantiate the effectiveness of both methods, as well as plug-and-play capability. We further analyze privacy preservation against memory extraction attacks, showing that while privacy leakage risk is independent of augmented public data ratio, it decreases or converges as training progresses.

Related papers

• Federated Cross-Domain Click-Through Rate Prediction With Large Language Model Augmentation [4.978132660177235]
  We present Federated Cross-Domain CTR Prediction with Large Language Model Augmentation (FedCCTR-LM) Our approach integrates three core innovations. First, the Privacy-Preserving Augmentation Network (PrivNet) employs large language models to enrich user and item representations. Second, the Independent Domain-Specific Transformer with Contrastive Learning (IDST-CL) module disentangles domain-specific and shared user preferences. Third, the Adaptive Local Differential Privacy (AdaLDP) mechanism dynamically calibrates noise injection to achieve an optimal balance between rigorous privacy guarantees and predictive accuracy.
  arXiv  Detail & Related papers  (2025-03-21T06:22:42Z)
• Privacy Preserving and Robust Aggregation for Cross-Silo Federated Learning in Non-IID Settings [1.8434042562191815]
  Federated Averaging remains the most widely used aggregation strategy in federated learning. Our method relies solely on gradient updates, eliminating the need for any additional client metadata. Our results establish the effectiveness of gradient masking as a practical and secure solution for federated learning.
  arXiv  Detail & Related papers  (2025-03-06T14:06:20Z)
• FedAlign: Federated Domain Generalization with Cross-Client Feature Alignment [2.4472081831862655]
  Federated Learning (FL) offers a decentralized paradigm for collaborative model training without direct data sharing. It poses unique challenges for Domain Generalization (DG), including strict privacy constraints, non-i.i.d. local data, and limited domain diversity. We introduce FedAlign, a lightweight, privacy-preserving framework designed to enhance DG in federated settings.
  arXiv  Detail & Related papers  (2025-01-26T11:17:32Z)
• Privacy Preserving Federated Unsupervised Domain Adaptation with Application to Age Prediction from DNA Methylation Data [2.699900017799093]
  We introduce a privacy-preserving framework for unsupervised domain adaptation in high-dimensional settings. Our framework is the first privacy-preserving solution for high-dimensional domain adaptation in federated environments.
  arXiv  Detail & Related papers  (2024-11-26T10:19:16Z)
• PARDON: Privacy-Aware and Robust Federated Domain Generalization [5.584498171854557]
  Federated Learning (FL) shows promise in preserving privacy and enabling collaborative learning. A significant challenge arises when client data comes from diverse domains, leading to poor performance on unseen domains. Existing Federated Domain Generalization approaches address this problem but assume each client holds data for an entire domain. We introduce FISC, a novel FedDG paradigm designed to robustly handle more complicated domain distributions.
  arXiv  Detail & Related papers  (2024-10-30T00:50:23Z)
• Enhancing Federated Domain Adaptation with Multi-Domain Prototype-Based Federated Fine-Tuning [15.640664498531274]
  Federated Domain Adaptation (FDA) is a Federated Learning (FL) scenario where models are trained across multiple clients without transmitting private data. We propose a novel framework called textbfMulti-domain textbfPrototype-based textbfFederated Fine-textbfTuning (MPFT) MPFT fine-tunes a pre-trained model using multi-domain prototypes, i.e., pretrained representations enriched with domain-specific information from category-specific local data.
  arXiv  Detail & Related papers  (2024-10-10T09:15:56Z)
• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• FewFedPIT: Towards Privacy-preserving and Few-shot Federated Instruction Tuning [54.26614091429253]
  Federated instruction tuning (FedIT) is a promising solution, by consolidating collaborative training across multiple data owners. FedIT encounters limitations such as scarcity of instructional data and risk of exposure to training data extraction attacks. We propose FewFedPIT, designed to simultaneously enhance privacy protection and model performance of federated few-shot learning.
  arXiv  Detail & Related papers  (2024-03-10T08:41:22Z)
• Federated Causal Discovery from Heterogeneous Data [70.31070224690399]
  We propose a novel FCD method attempting to accommodate arbitrary causal models and heterogeneous data. These approaches involve constructing summary statistics as a proxy of the raw data to protect data privacy. We conduct extensive experiments on synthetic and real datasets to show the efficacy of our method.
  arXiv  Detail & Related papers  (2024-02-20T18:53:53Z)
• Clients Collaborate: Flexible Differentially Private Federated Learning with Guaranteed Improvement of Utility-Privacy Trade-off [34.2117116062642]
  We introduce a novel federated learning framework with rigorous privacy guarantees, named FedCEO, to strike a trade-off between model utility and user privacy. We show that our FedCEO can effectively recover the disrupted semantic information by smoothing the global semantic space. It observes significant performance improvements and strict privacy guarantees under different privacy settings.
  arXiv  Detail & Related papers  (2024-02-10T17:39:34Z)
• FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation [95.85026305874824]
  We introduce a data-driven approach called FedSkip to improve the client optima by periodically skipping federated averaging and scattering local models to the cross devices. We conduct extensive experiments on a range of datasets to demonstrate that FedSkip achieves much higher accuracy, better aggregation efficiency and competing communication efficiency.
  arXiv  Detail & Related papers  (2022-12-14T13:57:01Z)
• Mitigating Data Heterogeneity in Federated Learning with Data Augmentation [26.226057709504733]
  Federated Learning (FL) is a framework that enables training a centralized model while securing user privacy by fusing local, decentralized models. One major obstacle is data heterogeneity, i.e., each client having non-identically and independently distributed (non-IID) data. Recent evidence suggests that data augmentation can induce equal or greater performance.
  arXiv  Detail & Related papers  (2022-06-20T19:47:43Z)
• Source-Free Domain Adaptation via Distribution Estimation [106.48277721860036]
  Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different. Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data. In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation.
  arXiv  Detail & Related papers  (2022-04-24T12:22:19Z)
• Federated Unsupervised Domain Adaptation for Face Recognition [26.336693850812118]
  We propose federated unsupervised domain adaptation for face recognition, FedFR. For unlabeled data in the target domain, we enhance a clustering algorithm with distance constrain to improve the quality of predicted pseudo labels. We also propose a new domain constraint loss to regularize source domain training in federated learning.
  arXiv  Detail & Related papers  (2022-04-09T04:02:03Z)
• Federated Multi-Target Domain Adaptation [99.93375364579484]
  Federated learning methods enable us to train machine learning models on distributed user data while preserving its privacy. We consider a more practical scenario where the distributed client data is unlabeled, and a centralized labeled dataset is available on the server. We propose an effective DualAdapt method to address the new challenges.
  arXiv  Detail & Related papers  (2021-08-17T17:53:05Z)
• Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy [67.4471689755097]
  This paper empirically demonstrates that the clipped FedAvg can perform surprisingly well even with substantial data heterogeneity. We provide the convergence analysis of a differential private (DP) FedAvg algorithm and highlight the relationship between clipping bias and the distribution of the clients' updates.
  arXiv  Detail & Related papers  (2021-06-25T14:47:19Z)
• Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation [74.71931918541748]
  We propose an instance affinity based criterion for source to target transfer during adaptation, called ILA-DA. We first propose a reliable and efficient method to extract similar and dissimilar samples across source and target, and utilize a multi-sample contrastive loss to drive the domain alignment process. We verify the effectiveness of ILA-DA by observing consistent improvements in accuracy over popular domain adaptation approaches on a variety of benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-03T01:33:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.