The Resource Problem of Using Linear Layer Leakage Attack in Federated Learning

• URL: http://arxiv.org/abs/2303.14868v1
• Date: Mon, 27 Mar 2023 01:21:31 GMT
• Title: The Resource Problem of Using Linear Layer Leakage Attack in Federated Learning
• Authors: Joshua C. Zhao, Ahmed Roushdy Elkordy, Atul Sharma, Yahya H. Ezzeldin, Salman Avestimehr, Saurabh Bagchi
• Abstract summary: We show that sparsity can decrease the model size overhead by over 327$times$ and the computation time by 3.34$times$ compared to SOTA. We show that the use of sparsity can decrease the model size overhead by over 327$times$ and the computation time by 3.34$times$ compared to SOTA.
• Score: 18.34693758013391
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Secure aggregation promises a heightened level of privacy in federated learning, maintaining that a server only has access to a decrypted aggregate update. Within this setting, linear layer leakage methods are the only data reconstruction attacks able to scale and achieve a high leakage rate regardless of the number of clients or batch size. This is done through increasing the size of an injected fully-connected (FC) layer. However, this results in a resource overhead which grows larger with an increasing number of clients. We show that this resource overhead is caused by an incorrect perspective in all prior work that treats an attack on an aggregate update in the same way as an individual update with a larger batch size. Instead, by attacking the update from the perspective that aggregation is combining multiple individual updates, this allows the application of sparsity to alleviate resource overhead. We show that the use of sparsity can decrease the model size overhead by over 327$\times$ and the computation time by 3.34$\times$ compared to SOTA while maintaining equivalent total leakage rate, 77% even with $1000$ clients in aggregation.

Related papers

• Harnessing Sparsification in Federated Learning: A Secure, Efficient, and Differentially Private Realization [28.546805212017926]
  Federated learning (FL) enables multiple clients to jointly train a model by sharing only gradient updates for aggregation instead of raw data.<n>We present Clover, a novel system framework for communication-efficient, secure, and differentially private FL.
  arXiv  Detail & Related papers  (2025-11-10T14:10:48Z)
• Enhancing Federated Learning Privacy with QUBO [0.0]
  Federated learning (FL) is a widely used method for training machine learning (ML) models in a scalable way while preserving privacy.<n>In this paper, we focus on two threat vectors: information leakage by clients during training and adversaries who can query or obtain the global model.<n> Experiments on the MNIST dataset with 300 clients in 20 rounds showed a 95.2% per-round and 49% cumulative privacy exposure reduction.
  arXiv  Detail & Related papers  (2025-11-04T18:06:30Z)
• LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging [80.17238673443127]
  LiNeS is a post-training editing technique designed to preserve pre-trained generalization while enhancing fine-tuned task performance. LiNeS demonstrates significant improvements in both single-task and multi-task settings across various benchmarks in vision and natural language processing.
  arXiv  Detail & Related papers  (2024-10-22T16:26:05Z)
• Heterogeneity-Aware Coordination for Federated Learning via Stitching Pre-trained blocks [5.621750660969172]
  Federated learning (FL) coordinates multiple devices to collaboratively train a shared model while preserving data privacy. We propose FedStitch, a hierarchical coordination framework for heterogeneous federated learning with pre-trained blocks.
  arXiv  Detail & Related papers  (2024-09-11T11:47:50Z)
• High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updates [50.406127962933915]
  We develop solutions to problems which enable us to learn a communication-efficient distributed logistic regression model. In our experiments we demonstrate a large improvement in accuracy over distributed algorithms with only a few distributed update steps needed.
  arXiv  Detail & Related papers  (2024-07-08T19:34:39Z)
• LOKI: Large-scale Data Reconstruction Attack against Federated Learning through Model Manipulation [25.03733882637947]
  We introduce LOKI, an attack that overcomes previous limitations and also breaks the anonymity of aggregation. With FedAVG and aggregation across 100 clients, prior work can leak less than 1% of images on MNIST, CIFAR-100, and Tiny ImageNet. Using only a single training round, LOKI is able to leak 76-86% of all data samples.
  arXiv  Detail & Related papers  (2023-03-21T23:29:35Z)
• Client-specific Property Inference against Secure Aggregation in Federated Learning [52.8564467292226]
  Federated learning has become a widely used paradigm for collaboratively training a common model among different participants. Many attacks have shown that it is still possible to infer sensitive information such as membership, property, or outright reconstruction of participant data. We show that simple linear models can effectively capture client-specific properties only from the aggregated model updates.
  arXiv  Detail & Related papers  (2023-03-07T14:11:01Z)
• Robust Quantity-Aware Aggregation for Federated Learning [72.59915691824624]
  Malicious clients can poison model updates and claim large quantities to amplify the impact of their model updates in the model aggregation. Existing defense methods for FL, while all handling malicious model updates, either treat all quantities benign or simply ignore/truncate the quantities of all clients. We propose a robust quantity-aware aggregation algorithm for federated learning, called FedRA, to perform the aggregation with awareness of local data quantities.
  arXiv  Detail & Related papers  (2022-05-22T15:13:23Z)
• TOFU: Towards Obfuscated Federated Updates by Encoding Weight Updates into Gradients from Proxy Data [7.489265323050362]
  We propose TOFU, a novel algorithm which generates proxy data that encodes the weight updates for each client in its gradients. We show that TOFU enables learning with less than 1% and 7% accuracy drops on MNIST and on CIFAR-10 datasets. This enables us to learn to near-full accuracy in a federated setup, while being 4x and 6.6x more communication efficient than the standard Federated Averaging algorithm.
  arXiv  Detail & Related papers  (2022-01-21T00:25:42Z)
• Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models [56.0250919557652]
  Federated learning has quickly gained popularity with its promises of increased user privacy and efficiency. Previous attacks on user privacy have been limited in scope and do not scale to gradient updates aggregated over even a handful of data points. We introduce a new threat model based on minimal but malicious modifications of the shared model architecture.
  arXiv  Detail & Related papers  (2021-10-25T15:52:06Z)
• FEDZIP: A Compression Framework for Communication-Efficient Federated Learning [2.334824705384299]
  Federated Learning is an implementation of decentralized machine learning for wireless devices. It assigns the learning process independently to each client. We propose a novel framework, FedZip, that significantly decreases the size of updates while transferring weights from the deep learning model between clients and their servers.
  arXiv  Detail & Related papers  (2021-02-02T16:33:44Z)
• Timely Communication in Federated Learning [65.1253801733098]
  We consider a global learning framework in which a parameter server (PS) trains a global model by using $n$ clients without actually storing the client data centrally at a cloud server. Under the proposed scheme, at each iteration, the PS waits for $m$ available clients and sends them the current model. We find the average age of information experienced by each client and numerically characterize the age-optimal $m$ and $k$ values for a given $n$.
  arXiv  Detail & Related papers  (2020-12-31T18:52:08Z)
• Sparse Communication for Training Deep Networks [56.441077560085475]
  Synchronous gradient descent (SGD) is the most common method used for distributed training of deep learning models. In this algorithm, each worker shares its local gradients with others and updates the parameters using the average gradients of all workers. We study several compression schemes and identify how three key parameters affect the performance.
  arXiv  Detail & Related papers  (2020-09-19T17:28:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.