Resource-Efficient Federated Fine-Tuning Large Language Models for Heterogeneous Data

• URL: http://arxiv.org/abs/2503.21213v1
• Date: Thu, 27 Mar 2025 07:05:22 GMT
• Title: Resource-Efficient Federated Fine-Tuning Large Language Models for Heterogeneous Data
• Authors: Jun Liu, Yunming Liao, Hongli Xu, Yang Xu,
• Abstract summary: Fine-tuning large language models (LLMs) via federated learning, i.e., FedLLM, has been proposed to adapt LLMs for various downstream applications in a privacy-preserving way.<n>To reduce the fine-tuning costs on resource-constrained devices, FedLoRA is proposed to fine-tune only a small subset of model parameters by integrating low-rank adaptation (LoRA) into FedLLM.<n>Here, we propose a hierarchical FedLoRA framework, termed HierFedLoRA, to address these challenges.
• Score: 16.844142562389443
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fine-tuning large language models (LLMs) via federated learning, i.e., FedLLM, has been proposed to adapt LLMs for various downstream applications in a privacy-preserving way. To reduce the fine-tuning costs on resource-constrained devices, FedLoRA is proposed to fine-tune only a small subset of model parameters by integrating low-rank adaptation (LoRA) into FedLLM. However, apart from resource constraints, there is still another critical challenge, i.e., data heterogeneity, severely hindering the implementation of FedLoRA in practical applications. Herein, inspired by the previous group-based federated learning paradigm, we propose a hierarchical FedLoRA framework, termed HierFedLoRA, to address these challenges. Specifically, HierFedLoRA partitions all devices into multiple near-IID groups and adjusts the intra-group aggregation frequency for each group to eliminate the negative effects of non-IID data. Meanwhile, to reduce the computation and communication cost, HierFedLoRA dynamically assigns diverse and suitable fine-tuning depth (i.e., the number of continuous fine-tuning layers from the output) for each group. HierFedLoRA explores jointly optimizing aggregation frequency and depth upon their coupled relationship to better enhance the performance of FedLoRA. Extensive experiments are conducted on a physical platform with 80 commercial devices. The results show that HierFedLoRA improves the final model accuracy by 1.6% to 4.2%, speeding up the fine-tuning process by at least 2.1$\times$, compared to the strong baselines.

Related papers

• BeamLoRA: Beam-Constraint Low-Rank Adaptation [51.52097743781401]
  Low-Rank Adaptation (LoRA) has been widely adopted as one of the most effective parameter-efficient fine-tuning methods.<n>We propose BeamLoRA, which conceptualizes each LoRA module as a beam where each rank naturally corresponds to a potential sub-solution.
  arXiv  Detail & Related papers  (2025-02-19T10:33:22Z)
• SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning [73.93639228235622]
  Continual Learning with foundation models has emerged as a promising paradigm to exploit abundant knowledge acquired during pre-training for tackling sequential tasks.<n>Existing prompt-based and Low-Rank Adaptation-based (LoRA-based) methods often require expanding a prompt/LoRA pool or retaining samples of previous tasks.<n>We propose Scalable Decoupled LoRA (SD-LoRA) for class incremental learning, which continually separates the learning of the magnitude and direction of LoRA components without rehearsal.
  arXiv  Detail & Related papers  (2025-01-22T20:00:41Z)
• Adaptive Parameter-Efficient Federated Fine-Tuning on Heterogeneous Devices [24.725928966071212]
  Federated fine-tuning (FedFT) has been proposed to fine-tune the pre-trained language models in a distributed manner.<n>We propose a novel LoRA-based FedFT framework, termed LEGEND, which faces the difficulty of determining the number of LoRA layers.<n>We analyze the coupled relationship between LoRA depth and rank distribution, and design an efficient LoRA configuration algorithm for heterogeneous devices.
  arXiv  Detail & Related papers  (2024-12-28T04:00:42Z)
• LoRA-FAIR: Federated LoRA Fine-Tuning with Aggregation and Initialization Refinement [5.162783756846019]
  Foundation models (FMs) achieve strong performance across diverse tasks with task-specific fine-tuning.<n>Low-Rank Adaptation (LoRA) methods like Low-Rank Adaptation (LoRA) reduce this cost by introducing low-rank matrices for tuning fewer parameters.<n>LoRA-FAIR maintains computational and communication efficiency, yielding superior performance over state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-11-22T14:19:01Z)
• Less is More: Extreme Gradient Boost Rank-1 Adaption for Efficient Finetuning of LLMs [75.11449420928139]
  Fine-tuning Large Language Models (LLMs) has become a crucial technique for adapting pre-trained models to downstream tasks. Low-Rank Adaptation (LoRA) has emerged as a promising solution, but there exists a gap between the practical performance of low-rank adaptations and its theoretical optimum. We propose eXtreme Gradient Boosting LoRA, a novel framework that bridges this gap by leveraging the power of ensemble learning.
  arXiv  Detail & Related papers  (2024-10-25T17:07:13Z)
• FedEx-LoRA: Exact Aggregation for Federated and Efficient Fine-Tuning of Foundation Models [5.1613368481802455]
  Low-Rank Adaptation (LoRA) is a popular technique for efficient fine-tuning of foundation models.<n>We propose Federated Exact LoRA, or FedEx-LoRA, which adds a residual error term to the pretrained frozen weight matrix.<n>Our approach achieves exact updates with minimal computational and communication overhead, preserving LoRA's efficiency.
  arXiv  Detail & Related papers  (2024-10-12T08:22:44Z)
• Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation [58.288682735160585]
  Low-Rank Adaptation (LoRA) is a popular technique for finetuning models. LoRA often under performs when compared to full- parameter fine-tuning. We present a framework that rigorously analyzes the adaptation rates of LoRA methods.
  arXiv  Detail & Related papers  (2024-10-10T18:51:53Z)
• FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations [39.88985198467528]
  We introduce a new approach called FLORA that enables federated fine-tuning on heterogeneous LoRA adapters. Our approach is noise-free and seamlessly supports heterogeneous LoRA adapters.
  arXiv  Detail & Related papers  (2024-09-09T18:21:23Z)
• Improving LoRA in Privacy-preserving Federated Learning [44.47315926976059]
  Low-rank adaptation (LoRA) is one of the most popular task-specific parameter-efficient fine-tuning (PEFT) methods on pre-trained language models. This paper proposes an efficient and effective version of LoRA, Federated Freeze A LoRA (FFA-LoRA), to alleviate these challenges.
  arXiv  Detail & Related papers  (2024-03-18T23:20:08Z)
• Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models [20.707283766914017]
  HetLoRA allows heterogeneous ranks across client devices and efficiently aggregates and distributes these heterogeneous LoRA modules. HetLoRA achieves improved convergence speed and final performance compared to homogeneous LoRA.
  arXiv  Detail & Related papers  (2024-01-12T07:52:07Z)
• FedRA: A Random Allocation Strategy for Federated Tuning to Unleash the Power of Heterogeneous Clients [50.13097183691517]
  In real-world federated scenarios, there often exist a multitude of heterogeneous clients with varying computation and communication resources. We propose a novel federated tuning algorithm, FedRA. In each communication round, FedRA randomly generates an allocation matrix. It reorganizes a small number of layers from the original model based on the allocation matrix and fine-tunes using adapters.
  arXiv  Detail & Related papers  (2023-11-19T04:43:16Z)
• FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning [70.38817963253034]
  This paper first discusses these challenges of federated fine-tuning LLMs, and introduces our package FS-LLM as a main contribution. We provide comprehensive federated parameter-efficient fine-tuning algorithm implementations and versatile programming interfaces for future extension in FL scenarios. We conduct extensive experiments to validate the effectiveness of FS-LLM and benchmark advanced LLMs with state-of-the-art parameter-efficient fine-tuning algorithms in FL settings.
  arXiv  Detail & Related papers  (2023-09-01T09:40:36Z)
• Heterogeneous Federated Learning via Grouped Sequential-to-Parallel Training [60.892342868936865]
  Federated learning (FL) is a rapidly growing privacy-preserving collaborative machine learning paradigm. We propose a data heterogeneous-robust FL approach, FedGSP, to address this challenge. We show that FedGSP improves the accuracy by 3.7% on average compared with seven state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-01-31T03:15:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.