Research on Large Language Model Cross-Cloud Privacy Protection and Collaborative Training based on Federated Learning

• URL: http://arxiv.org/abs/2503.12226v1
• Date: Sat, 15 Mar 2025 18:44:50 GMT
• Title: Research on Large Language Model Cross-Cloud Privacy Protection and Collaborative Training based on Federated Learning
• Authors: Ze Yang, Yihong Jin, Yihan Zhang, Juntian Liu, Xinhe Xu,
• Abstract summary: Large language models (LLMs) and popularization of cloud computing have led to increasing concerns on privacy safeguarding and data security of cross-cloud model deployment and training.<n>We present a new framework for addressing these issues along with enabling privacy preserving collaboration on training between distributed clouds based on federated learning.
• Score: 5.853391005435494
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The fast development of large language models (LLMs) and popularization of cloud computing have led to increasing concerns on privacy safeguarding and data security of cross-cloud model deployment and training as the key challenges. We present a new framework for addressing these issues along with enabling privacy preserving collaboration on training between distributed clouds based on federated learning. Our mechanism encompasses cutting-edge cryptographic primitives, dynamic model aggregation techniques, and cross-cloud data harmonization solutions to enhance security, efficiency, and scalability to the traditional federated learning paradigm. Furthermore, we proposed a hybrid aggregation scheme to mitigate the threat of Data Leakage and to optimize the aggregation of model updates, thus achieving substantial enhancement on the model effectiveness and stability. Experimental results demonstrate that the training efficiency, privacy protection, and model accuracy of the proposed model compare favorably to those of the traditional federated learning method.

Related papers

• Advancing Personalized Federated Learning: Integrative Approaches with AI for Enhanced Privacy and Customization [0.0]
  This paper proposes a novel approach that enhances PFL with cutting-edge AI techniques.<n>We present a model that boosts the performance of individual client models and ensures robust privacy-preserving mechanisms.<n>This work paves the way for a new era of truly personalized and privacy-conscious AI systems.
  arXiv  Detail & Related papers  (2025-01-30T07:03:29Z)
• DWFL: Enhancing Federated Learning through Dynamic Weighted Averaging [2.499907423888049]
  We propose a deep feed-forward neural network-based enhanced federated learning method for protein sequence classification. We introduce dynamic weighted federated learning (DWFL) which is a federated learning-based approach. We conduct experiments using real-world protein sequence datasets to assess the effectiveness of DWFL.
  arXiv  Detail & Related papers  (2024-11-07T20:24:23Z)
• Research on Key Technologies for Cross-Cloud Federated Training of Large Language Models [7.762524368844918]
  Cross-cloud federated training offers a new approach to addressing the resource bottlenecks of a single cloud platform.<n>This study analyzes the key technologies of cross-cloud federated training, including data partitioning and distribution, communication optimization, model aggregation algorithms, and the compatibility of heterogeneous cloud platforms.
  arXiv  Detail & Related papers  (2024-10-24T19:57:17Z)
• Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models [52.03511469562013]
  We introduce the Iterative Contrastive Unlearning (ICU) framework, which consists of three core components.<n>A Knowledge Unlearning Induction module targets specific knowledge for removal using an unlearning loss.<n>A Contrastive Learning Enhancement module preserves the model's expressive capabilities against the pure unlearning goal.<n>An Iterative Unlearning Refinement module dynamically adjusts the unlearning process through ongoing evaluation and updates.
  arXiv  Detail & Related papers  (2024-07-25T07:09:35Z)
• PPBFL: A Privacy Protected Blockchain-based Federated Learning Model [6.278098707317501]
  We propose a Protected-based Federated Learning Model (PPBFL) to enhance the security of federated learning. We introduce a Proof of Training Work (PoTW) algorithm tailored for federated learning, aiming to incentive training nodes. We also propose a new mix transactions mechanism utilizing ring signature technology to better protect the identity privacy of local training clients.
  arXiv  Detail & Related papers  (2024-01-02T13:13:28Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• FRAMU: Attention-based Machine Unlearning using Federated Reinforcement Learning [16.86560475992975]
  We introduce Attention-based Machine Unlearning using Federated Reinforcement Learning (FRAMU) FRAMU incorporates adaptive learning mechanisms, privacy preservation techniques, and optimization strategies. Our experiments, conducted on both single-modality and multi-modality datasets, revealed that FRAMU significantly outperformed baseline models.
  arXiv  Detail & Related papers  (2023-09-19T03:13:17Z)
• Federated Learning-Empowered AI-Generated Content in Wireless Networks [58.48381827268331]
  Federated learning (FL) can be leveraged to improve learning efficiency and achieve privacy protection for AIGC. We present FL-based techniques for empowering AIGC, and aim to enable users to generate diverse, personalized, and high-quality content.
  arXiv  Detail & Related papers  (2023-07-14T04:13:11Z)
• Towards a Better Theoretical Understanding of Independent Subnetwork Training [56.24689348875711]
  We take a closer theoretical look at Independent Subnetwork Training (IST) IST is a recently proposed and highly effective technique for solving the aforementioned problems. We identify fundamental differences between IST and alternative approaches, such as distributed methods with compressed communication.
  arXiv  Detail & Related papers  (2023-06-28T18:14:22Z)
• Personalizing Federated Learning with Over-the-Air Computations [84.8089761800994]
  Federated edge learning is a promising technology to deploy intelligence at the edge of wireless networks in a privacy-preserving manner. Under such a setting, multiple clients collaboratively train a global generic model under the coordination of an edge server. This paper presents a distributed training paradigm that employs analog over-the-air computation to address the communication bottleneck.
  arXiv  Detail & Related papers  (2023-02-24T08:41:19Z)
• Delving into Data: Effectively Substitute Training for Black-box Attack [84.85798059317963]
  We propose a novel perspective substitute training that focuses on designing the distribution of data used in the knowledge stealing process. The combination of these two modules can further boost the consistency of the substitute model and target model, which greatly improves the effectiveness of adversarial attack.
  arXiv  Detail & Related papers  (2021-04-26T07:26:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.