FedAttr: Towards Privacy-preserving Client-Level Attribution in Federated LLM Fine-tuning

• URL: http://arxiv.org/abs/2605.06596v1
• Date: Thu, 07 May 2026 17:21:02 GMT
• Title: FedAttr: Towards Privacy-preserving Client-Level Attribution in Federated LLM Fine-tuning
• Abstract summary: We propose FedAttr, a new client-level attribution protocol for federated learning.<n>FedAttr identifies which clients trained on watermarked data via a paired-subset-difference mechanism.<n>It achieves 100% TPR and 0% FPR, outperforming all baselines by at least 44.4% in TPR or 19.1% in FPR, with only 6.3% overhead relative to FL training time.
• Score: 58.80511162314259
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Watermark radioactivity testing type of methods can detect whether a model was trained on watermarked documents, and have become key tools for protecting data ownership in the fine-tuning of large language models (LLMs). Existing works have proved their effectiveness in centralized LLM fine-tuning. However, this type of method faces several challenges and remains underexplored in federated learning (FL), a widely-applied paradigm for fine-tuning LLMs collaboratively on private data across different users. FL mainly ensures privacy through secure aggregation (SA), which allows the server to aggregate updates while keeping clients' updates private. This mechanism preserves privacy but makes it difficult to identify which client trained on watermarked documents. In this work, we propose FedAttr, a new client-level attribution protocol for FL. FedAttr identifies which clients trained on watermarked data via a paired-subset-difference mechanism, while preserving the privacy guarantees of SA and FL performance. FedAttr proceeds in three steps: (i) estimate each client's update by differencing two SA queries, (ii) score the estimate with the watermark detector via differential scoring, and (iii) combine scores across rounds via Stouffer method. We theoretically show that FedAttr produces an unbiased estimator of each client's update with bounded mutual information leakage (i.e., $O(d^*/N)$ per-round update). Moreover, FedAttr empirically achieves 100% TPR and 0% FPR, outperforming all baselines by at least 44.4% in TPR or 19.1% in FPR, with only 6.3% overhead relative to FL training time. Ablation studies confirm that FedAttr is robust to protocol parameters and configurations.

Related papers

• FedFG: Privacy-Preserving and Robust Federated Learning via Flow-Matching Generation [7.120183076762444]
  Federated learning (FL) enables distributed clients to collaboratively train a global model using local private data.<n>Recent studies show that conventional FL algorithms still exhibit deficiencies in privacy protection.<n>We propose FedFG, a robust FL framework based on flow-matching generation that simultaneously preserves client privacy and resists sophisticated poisoning attacks.
  arXiv  Detail & Related papers  (2026-03-30T03:11:35Z)
• Local Data Quantity-Aware Weighted Averaging for Federated Learning with Dishonest Clients [32.11272564261484]
  Federated learning (FL) enables collaborative training of deep learning models without requiring data to leave local clients.<n>Most commonly used aggregation method is weighted averaging based on the amount of data from each client, which is thought to reflect each client's contribution.<n>We propose a novel secure underlineFederated underlineData qunderlineuantity-underlineaware weighted averaging method (FedDua)<n>It enables FL servers to accurately predict the amount of training data from each client based on their local model gradients uploaded.
  arXiv  Detail & Related papers  (2025-04-17T01:50:24Z)
• FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA Subparameter Updates [58.18162789618869]
  Federated Learning (FL) is a widely used framework for training models in a decentralized manner.<n>We propose the FedRand framework, which avoids disclosing the full set of client parameters.<n>We empirically validate that FedRand improves robustness against MIAs compared to relevant baselines.
  arXiv  Detail & Related papers  (2025-03-10T11:55:50Z)
• FedMIA: An Effective Membership Inference Attack Exploiting "All for One" Principle in Federated Learning [17.141646895576145]
  Federated Learning (FL) is a promising approach for training machine learning models on decentralized data.<n>Membership Inference Attacks (MIAs) aim to determine whether a specific data point belongs to a target client's training set.<n>We introduce a three-step Membership Inference Attack (MIA) method, called FedMIA, which follows the "all for one"--leveraging updates from all clients across multiple communication rounds to enhance MIA effectiveness.
  arXiv  Detail & Related papers  (2024-02-09T09:58:35Z)
• Federated Nearest Neighbor Machine Translation [66.8765098651988]
  In this paper, we propose a novel federated nearest neighbor (FedNN) machine translation framework. FedNN leverages one-round memorization-based interaction to share knowledge across different clients. Experiments show that FedNN significantly reduces computational and communication costs compared with FedAvg.
  arXiv  Detail & Related papers  (2023-02-23T18:04:07Z)
• Scalable Collaborative Learning via Representation Sharing [53.047460465980144]
  Federated learning (FL) and Split Learning (SL) are two frameworks that enable collaborative learning while keeping the data private (on device) In FL, each data holder trains a model locally and releases it to a central server for aggregation. In SL, the clients must release individual cut-layer activations (smashed data) to the server and wait for its response (during both inference and back propagation). In this work, we present a novel approach for privacy-preserving machine learning, where the clients collaborate via online knowledge distillation using a contrastive loss.
  arXiv  Detail & Related papers  (2022-11-20T10:49:22Z)
• FedFM: Anchor-based Feature Matching for Data Heterogeneity in Federated Learning [91.74206675452888]
  We propose a novel method FedFM, which guides each client's features to match shared category-wise anchors. To achieve higher efficiency and flexibility, we propose a FedFM variant, called FedFM-Lite, where clients communicate with server with fewer synchronization times and communication bandwidth costs.
  arXiv  Detail & Related papers  (2022-10-14T08:11:34Z)
• Cross-client Label Propagation for Transductive and Semi-Supervised Federated Learning [18.422153315741134]
  XCLP estimates a data graph jointly from the data of multiple clients and computes labels for the unlabeled data by propagating label information across the graph. To avoid clients having to share their data with anyone, XCLP employs two cryptographically secure protocols.
  arXiv  Detail & Related papers  (2022-10-12T17:33:47Z)
• FedPerm: Private and Robust Federated Learning by Parameter Permutation [2.406359246841227]
  Federated Learning (FL) is a distributed learning paradigm that enables mutually untrusting clients to collaboratively train a common machine learning model. Client data privacy is paramount in FL. At the same time, the model must be protected from poisoning attacks from adversarial clients. We present FedPerm, a new FL algorithm that addresses both these problems by combining a novel intra-model parameter shuffling technique that amplifies data privacy, with Private Information Retrieval (PIR) based techniques that permit cryptographic aggregation of clients' model updates.
  arXiv  Detail & Related papers  (2022-08-16T19:40:28Z)
• Stochastic Coded Federated Learning with Convergence and Privacy Guarantees [8.2189389638822]
  Federated learning (FL) has attracted much attention as a privacy-preserving distributed machine learning framework. This paper proposes a coded federated learning framework, namely coded federated learning (SCFL) to mitigate the straggler issue. We characterize the privacy guarantee by the mutual information differential privacy (MI-DP) and analyze the convergence performance in federated learning.
  arXiv  Detail & Related papers  (2022-01-25T04:43:29Z)
• Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy [67.4471689755097]
  This paper empirically demonstrates that the clipped FedAvg can perform surprisingly well even with substantial data heterogeneity. We provide the convergence analysis of a differential private (DP) FedAvg algorithm and highlight the relationship between clipping bias and the distribution of the clients' updates.
  arXiv  Detail & Related papers  (2021-06-25T14:47:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.