FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large
Language Models in Federated Learning
- URL: http://arxiv.org/abs/2309.00363v1
- Date: Fri, 1 Sep 2023 09:40:36 GMT
- Title: FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large
Language Models in Federated Learning
- Authors: Weirui Kuang, Bingchen Qian, Zitao Li, Daoyuan Chen, Dawei Gao, Xuchen
Pan, Yuexiang Xie, Yaliang Li, Bolin Ding, Jingren Zhou
- Abstract summary: This paper first discusses these challenges of federated fine-tuning LLMs, and introduces our package FS-LLM as a main contribution.
We provide comprehensive federated parameter-efficient fine-tuning algorithm implementations and versatile programming interfaces for future extension in FL scenarios.
We conduct extensive experiments to validate the effectiveness of FS-LLM and benchmark advanced LLMs with state-of-the-art parameter-efficient fine-tuning algorithms in FL settings.
- Score: 70.38817963253034
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: LLMs have demonstrated great capabilities in various NLP tasks. Different
entities can further improve the performance of those LLMs on their specific
downstream tasks by fine-tuning LLMs. When several entities have similar
interested tasks, but their data cannot be shared because of privacy concerns
regulations, federated learning (FL) is a mainstream solution to leverage the
data of different entities. However, fine-tuning LLMs in federated learning
settings still lacks adequate support from existing FL frameworks because it
has to deal with optimizing the consumption of significant communication and
computational resources, data preparation for different tasks, and distinct
information protection demands. This paper first discusses these challenges of
federated fine-tuning LLMs, and introduces our package FS-LLM as a main
contribution, which consists of the following components: (1) we build an
end-to-end benchmarking pipeline, automizing the processes of dataset
preprocessing, federated fine-tuning execution, and performance evaluation on
federated LLM fine-tuning; (2) we provide comprehensive federated
parameter-efficient fine-tuning algorithm implementations and versatile
programming interfaces for future extension in FL scenarios with low
communication and computation costs, even without accessing the full model; (3)
we adopt several accelerating and resource-efficient operators for fine-tuning
LLMs with limited resources and the flexible pluggable sub-routines for
interdisciplinary study. We conduct extensive experiments to validate the
effectiveness of FS-LLM and benchmark advanced LLMs with state-of-the-art
parameter-efficient fine-tuning algorithms in FL settings, which also yields
valuable insights into federated fine-tuning LLMs for the research community.
To facilitate further research and adoption, we release FS-LLM at
https://github.com/alibaba/FederatedScope/tree/llm.
Related papers
- LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularization [59.75242204923353]
We introduce LLM-Lasso, a framework that leverages large language models (LLMs) to guide feature selection in Lasso regression.
LLMs generate penalty factors for each feature, which are converted into weights for the Lasso penalty using a simple, tunable model.
Features identified as more relevant by the LLM receive lower penalties, increasing their likelihood of being retained in the final model.
arXiv Detail & Related papers (2025-02-15T02:55:22Z) - Federated Fine-Tuning of LLMs: Framework Comparison and Research Directions [59.5243730853157]
Federated learning (FL) provides a privacy-preserving solution for fine-tuning pre-trained large language models (LLMs) using distributed private datasets.
This article conducts a comparative analysis of three advanced federated LLM (FedLLM) frameworks that integrate knowledge distillation (KD) and split learning (SL) to mitigate these issues.
arXiv Detail & Related papers (2025-01-08T11:37:06Z) - Federated In-Context LLM Agent Learning [3.4757641432843487]
Large Language Models (LLMs) have revolutionized intelligent services by enabling logical reasoning, tool use, and interaction with external systems as agents.
In this paper, we propose a novel privacy-preserving Federated In-context LLM Agent Learning (FICAL) algorithm.
The results show that FICAL has competitive performance compared to other SOTA baselines with a significant communication cost decrease of $mathbf3.33times105$ times.
arXiv Detail & Related papers (2024-12-11T03:00:24Z) - The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities [0.35998666903987897]
This report examines the fine-tuning of Large Language Models (LLMs)
It outlines the historical evolution of LLMs from traditional Natural Language Processing (NLP) models to their pivotal role in AI.
The report introduces a structured seven-stage pipeline for fine-tuning LLMs.
arXiv Detail & Related papers (2024-08-23T14:48:02Z) - ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency [20.33467627548677]
Large language models (LLMs) have surged in popularity and are extensively used in commercial applications.
We conduct a detailed analysis to identify major bottlenecks that impact end-to-end latency in LLM serving systems.
We then propose ScaleLLM, an optimized system for resource-efficient LLM serving.
arXiv Detail & Related papers (2024-07-23T23:37:29Z) - Efficient Prompting for LLM-based Generative Internet of Things [88.84327500311464]
Large language models (LLMs) have demonstrated remarkable capacities on various tasks, and integrating the capacities of LLMs into the Internet of Things (IoT) applications has drawn much research attention recently.
Due to security concerns, many institutions avoid accessing state-of-the-art commercial LLM services, requiring the deployment and utilization of open-source LLMs in a local network setting.
We propose a LLM-based Generative IoT (GIoT) system deployed in the local network setting in this study.
arXiv Detail & Related papers (2024-06-14T19:24:00Z) - A Practice-Friendly LLM-Enhanced Paradigm with Preference Parsing for Sequential Recommendation [15.153844486572932]
This paper proposes a practice-friendly LLM-enhanced paradigm with preference parsing (P2Rec) for sequential recommender systems (SRS)
Specifically, in the information reconstruction stage, we design a new user-level SFT task for collaborative information injection with the assistance of a pre-trained SRS model.
Our goal is to let LLM learn to reconstruct a corresponding prior preference distribution from each user's interaction sequence.
arXiv Detail & Related papers (2024-06-01T07:18:56Z) - Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration [70.09561665520043]
We propose a novel framework for multi-agent collaboration that introduces Reinforced Advantage feedback (ReAd) for efficient self-refinement of plans.
We provide theoretical analysis by extending advantage-weighted regression in reinforcement learning to multi-agent systems.
Experiments on Over-AI and a difficult variant of RoCoBench show that ReAd surpasses baselines in success rate, and also significantly decreases the interaction steps of agents.
arXiv Detail & Related papers (2024-05-23T08:33:19Z) - Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes [53.4856038354195]
Pre-trained large language models (LLMs) need fine-tuning to improve their responsiveness to natural language instructions.
FedKSeed employs zeroth-order optimization with a finite set of random seeds.
It significantly reduces transmission requirements between the server and clients to just a few random seeds.
arXiv Detail & Related papers (2023-12-11T13:03:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.