Federated Large Language Models: Feasibility, Robustness, Security and Future Directions

• URL: http://arxiv.org/abs/2505.08830v1
• Date: Tue, 13 May 2025 03:23:54 GMT
• Title: Federated Large Language Models: Feasibility, Robustness, Security and Future Directions
• Authors: Wenhao Jiang, Yuchuan Luo, Guilin Deng, Silong Chen, Xu Yang, Shihong Wu, Xinwen Gao, Lin Liu, Shaojing Fu,
• Abstract summary: The integration of Large Language Models (LLMs) and Federated Learning (FL) presents a promising solution for joint training on distributed data.<n>This paper examines challenges from four critical perspectives: feasibility, robustness, security, and future directions.
• Score: 18.21433627004719
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The integration of Large Language Models (LLMs) and Federated Learning (FL) presents a promising solution for joint training on distributed data while preserving privacy and addressing data silo issues. However, this emerging field, known as Federated Large Language Models (FLLM), faces significant challenges, including communication and computation overheads, heterogeneity, privacy and security concerns. Current research has primarily focused on the feasibility of FLLM, but future trends are expected to emphasize enhancing system robustness and security. This paper provides a comprehensive review of the latest advancements in FLLM, examining challenges from four critical perspectives: feasibility, robustness, security, and future directions. We present an exhaustive survey of existing studies on FLLM feasibility, introduce methods to enhance robustness in the face of resource, data, and task heterogeneity, and analyze novel risks associated with this integration, including privacy threats and security challenges. We also review the latest developments in defense mechanisms and explore promising future research directions, such as few-shot learning, machine unlearning, and IP protection. This survey highlights the pressing need for further research to enhance system robustness and security while addressing the unique challenges posed by the integration of FL and LLM.

Related papers

• A Survey on Data Security in Large Language Models [12.23432845300652]
  Large Language Models (LLMs) are a foundation in advancing natural language processing, power applications such as text generation, machine translation, and conversational systems.<n>Despite their transformative potential, these models inherently rely on massive amounts of training data, often collected from diverse and uncurated sources, which exposes them to serious data security risks.<n>Harmful or malicious data can compromise model behavior, leading to issues such as toxic output, hallucinations, and vulnerabilities to threats such as prompt injection or data poisoning.<n>This survey offers a comprehensive overview of the main data security risks facing LLMs and reviews current defense strategies, including adversarial
  arXiv  Detail & Related papers  (2025-08-04T11:28:34Z)
• Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metrics [0.7481505949203433]
  Large Language Models (LLMs) have emerged as a promising cornerstone for the development of natural language processing (NLP) and artificial intelligence (AI)<n>This survey provides a comprehensive overview of current studies in this area.
  arXiv  Detail & Related papers  (2025-05-24T11:50:52Z)
• A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations [127.52707312573791]
  This survey provides a comprehensive analysis of LVLM safety, covering key aspects such as attacks, defenses, and evaluation methods.<n>We introduce a unified framework that integrates these interrelated components, offering a holistic perspective on the vulnerabilities of LVLMs.<n>We conduct a set of safety evaluations on the latest LVLM, Deepseek Janus-Pro, and provide a theoretical analysis of the results.
  arXiv  Detail & Related papers  (2025-02-14T08:42:43Z)
• Ten Challenging Problems in Federated Foundation Models [55.343738234307544]
  Federated Foundation Models (FedFMs) represent a distributed learning paradigm that fuses general competences of foundation models as well as privacy-preserving capabilities of federated learning.<n>This paper provides a comprehensive summary of the ten challenging problems inherent in FedFMs, encompassing foundational theory, utilization of private data, continual learning, unlearning, Non-IID and graph data, bidirectional knowledge transfer, incentive mechanism design, game mechanism design, model watermarking, and efficiency.
  arXiv  Detail & Related papers  (2025-02-14T04:01:15Z)
• Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety [296.5392512998251]
  We present a comprehensive taxonomy of safety threats to large models, including adversarial attacks, data poisoning, backdoor attacks, jailbreak and prompt injection attacks, energy-latency attacks, data and model extraction attacks, and emerging agent-specific threats.<n>We identify and discuss the open challenges in large model safety, emphasizing the need for comprehensive safety evaluations, scalable and effective defense mechanisms, and sustainable data practices.
  arXiv  Detail & Related papers  (2025-02-02T05:14:22Z)
• New Emerged Security and Privacy of Pre-trained Model: a Survey and Outlook [54.24701201956833]
  Security and privacy issues have undermined users' confidence in pre-trained models. Current literature lacks a clear taxonomy of emerging attacks and defenses for pre-trained models. This taxonomy categorizes attacks and defenses into No-Change, Input-Change, and Model-Change approaches.
  arXiv  Detail & Related papers  (2024-11-12T10:15:33Z)
• Trustworthy Federated Learning: Privacy, Security, and Beyond [37.495790989584584]
  Federated Learning (FL) addresses concerns by facilitating collaborative model training across distributed data sources without transferring raw data. We conduct an extensive survey of the security and privacy issues prevalent in FL, underscoring the vulnerability of communication links and the potential for cyber threats. We identify the intricate security challenges that arise within the FL frameworks, aiming to contribute to the development of secure and efficient FL systems.
  arXiv  Detail & Related papers  (2024-11-03T14:18:01Z)
• Federated Large Language Models: Current Progress and Future Directions [63.68614548512534]
  This paper surveys Federated learning for LLMs (FedLLM), highlighting recent advances and future directions. We focus on two key aspects: fine-tuning and prompt learning in a federated setting, discussing existing work and associated research challenges.
  arXiv  Detail & Related papers  (2024-09-24T04:14:33Z)
• Position Paper: Assessing Robustness, Privacy, and Fairness in Federated Learning Integrated with Foundation Models [39.86957940261993]
  Integration of Foundation Models (FMs) into Federated Learning (FL) introduces novel issues in terms of robustness, privacy, and fairness. We analyze the trade-offs involved, uncover the threats and issues introduced by this integration, and propose a set of criteria and strategies for navigating these challenges.
  arXiv  Detail & Related papers  (2024-02-02T19:26:00Z)
• Security and Privacy Challenges of Large Language Models: A Survey [2.6986500640871482]
  Large Language Models (LLMs) have demonstrated extraordinary capabilities and contributed to multiple fields, such as generating and summarizing text, language translation, and question-answering. These models are also vulnerable to security and privacy attacks, such as jailbreaking attacks, data poisoning attacks, and Personally Identifiable Information (PII) leakage attacks. This survey provides a thorough review of the security and privacy challenges of LLMs for both training data and users, along with the application-based risks in various domains, such as transportation, education, and healthcare.
  arXiv  Detail & Related papers  (2024-01-30T04:00:54Z)
• Exploring Federated Unlearning: Analysis, Comparison, and Insights [101.64910079905566]
  federated unlearning enables the selective removal of data from models trained in federated systems.<n>This paper examines existing federated unlearning approaches, examining their algorithmic efficiency, impact on model accuracy, and effectiveness in preserving privacy.<n>We propose the OpenFederatedUnlearning framework, a unified benchmark for evaluating federated unlearning methods.
  arXiv  Detail & Related papers  (2023-10-30T01:34:33Z)
• A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacy [47.89042524852868]
  Federated Learning (FL) stands out as a promising solution for diverse real-world scenarios. However, challenges around data isolation and privacy threaten the trustworthiness of FL systems.
  arXiv  Detail & Related papers  (2023-02-21T12:52:12Z)
• New Challenges in Reinforcement Learning: A Survey of Security and Privacy [26.706957408693363]
  Reinforcement learning (RL) is one of the most important branches of AI. RL has been widely applied in multiple areas, such as healthcare, data markets, autonomous driving, and robotics. Some of these applications and systems have been shown to be vulnerable to security or privacy attacks.
  arXiv  Detail & Related papers  (2022-12-31T12:30:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.