Federated Instruction Tuning of LLMs with Domain Coverage Augmentation

• URL: http://arxiv.org/abs/2409.20135v5
• Date: Tue, 21 Jan 2025 09:25:25 GMT
• Title: Federated Instruction Tuning of LLMs with Domain Coverage Augmentation
• Authors: Zezhou Wang, Yaxin Du, Xingjun Ma, Yugang Jiang, Zhuzhong Qian, Siheng Chen,
• Abstract summary: Federated Domain-specific Instruction Tuning (FedDIT) utilizes limited cross-client private data together with various strategies of instruction augmentation. We propose FedDCA, which optimize domain coverage through greedy client center selection and retrieval-based augmentation. For client-side computational efficiency and system scalability, FedDCA$*$, the variant of FedDCA, utilizes heterogeneous encoders with server-side feature alignment.
• Score: 87.49293964617128
• License:
• Abstract: Federated Domain-specific Instruction Tuning (FedDIT) utilizes limited cross-client private data together with various strategies of instruction augmentation, ultimately boosting model performance within specific domains. To date, the factors affecting FedDIT remain unclear, and existing instruction augmentation methods primarily focus on the centralized setting without considering distributed environments. Our experiments reveal that the cross-client domain coverage, rather than data heterogeneity, drives model performance in FedDIT. In response, we propose FedDCA, which optimizes domain coverage through greedy client center selection and retrieval-based augmentation. At its core, the greedy selection procedure iteratively picks client centers that maximize the diversity and coverage of the instruction space while avoiding redundancy with previously selected centers. This ensures broad yet efficient coverage of the domain distribution across clients. For client-side computational efficiency and system scalability, FedDCA$^*$, the variant of FedDCA, utilizes heterogeneous encoders with server-side feature alignment. Extensive experiments across code, medical, financial, and mathematical domains substantiate the effectiveness of both methods, as well as plug-and-play capability. We further analyze privacy preservation against memory extraction attacks, showing that while privacy leakage risk is independent of augmented public data ratio, it decreases or converges as training progresses.

Related papers

• Secure Federated Data Distillation [2.5311562666866494]
  We propose a Secure Federated Data Distillation framework (SFDD) to decentralize the distillation process while preserving privacy. Unlike existing Federated Distillation techniques that focus on training global models with distilled knowledge, our approach aims to produce a distilled dataset without exposing local contributions.
  arXiv  Detail & Related papers  (2025-02-19T13:54:44Z)
• FedAlign: Federated Domain Generalization with Cross-Client Feature Alignment [2.4472081831862655]
  Federated Learning (FL) offers a decentralized paradigm for collaborative model training without direct data sharing. It poses unique challenges for Domain Generalization (DG), including strict privacy constraints, non-i.i.d. local data, and limited domain diversity. We introduce FedAlign, a lightweight, privacy-preserving framework designed to enhance DG in federated settings.
  arXiv  Detail & Related papers  (2025-01-26T11:17:32Z)
• Boosting Federated Learning with FedEntOpt: Mitigating Label Skew by Entropy-Based Client Selection [13.851391819710367]
  Deep learning domains typically require an extensive amount of data for optimal performance. FedEntOpt is designed to mitigate performance issues caused by label distribution skew. It exhibits robust and superior performance in scenarios with low participation rates and client dropout.
  arXiv  Detail & Related papers  (2024-11-02T13:31:36Z)
• Federated Learning for Sparse Principal Component Analysis [0.0]
  Federated learning is a decentralized approach where model training occurs on client sides, preserving privacy by keeping data localized. We apply this framework to Sparse Principal Component Analysis (SPCA) in this work. SPCA aims to attain sparse component loadings while maximizing data variance for improved interpretability.
  arXiv  Detail & Related papers  (2023-11-15T03:55:28Z)
• FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation [95.85026305874824]
  We introduce a data-driven approach called FedSkip to improve the client optima by periodically skipping federated averaging and scattering local models to the cross devices. We conduct extensive experiments on a range of datasets to demonstrate that FedSkip achieves much higher accuracy, better aggregation efficiency and competing communication efficiency.
  arXiv  Detail & Related papers  (2022-12-14T13:57:01Z)
• Mitigating Data Heterogeneity in Federated Learning with Data Augmentation [26.226057709504733]
  Federated Learning (FL) is a framework that enables training a centralized model while securing user privacy by fusing local, decentralized models. One major obstacle is data heterogeneity, i.e., each client having non-identically and independently distributed (non-IID) data. Recent evidence suggests that data augmentation can induce equal or greater performance.
  arXiv  Detail & Related papers  (2022-06-20T19:47:43Z)
• Source-Free Domain Adaptation via Distribution Estimation [106.48277721860036]
  Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different. Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data. In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation.
  arXiv  Detail & Related papers  (2022-04-24T12:22:19Z)
• Federated Multi-Target Domain Adaptation [99.93375364579484]
  Federated learning methods enable us to train machine learning models on distributed user data while preserving its privacy. We consider a more practical scenario where the distributed client data is unlabeled, and a centralized labeled dataset is available on the server. We propose an effective DualAdapt method to address the new challenges.
  arXiv  Detail & Related papers  (2021-08-17T17:53:05Z)
• Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy [67.4471689755097]
  This paper empirically demonstrates that the clipped FedAvg can perform surprisingly well even with substantial data heterogeneity. We provide the convergence analysis of a differential private (DP) FedAvg algorithm and highlight the relationship between clipping bias and the distribution of the clients' updates.
  arXiv  Detail & Related papers  (2021-06-25T14:47:19Z)
• Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation [74.71931918541748]
  We propose an instance affinity based criterion for source to target transfer during adaptation, called ILA-DA. We first propose a reliable and efficient method to extract similar and dissimilar samples across source and target, and utilize a multi-sample contrastive loss to drive the domain alignment process. We verify the effectiveness of ILA-DA by observing consistent improvements in accuracy over popular domain adaptation approaches on a variety of benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-03T01:33:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.