Cocktail Party Attack: Breaking Aggregation-Based Privacy in Federated Learning using Independent Component Analysis

• URL: http://arxiv.org/abs/2209.05578v1
• Date: Mon, 12 Sep 2022 20:01:53 GMT
• Title: Cocktail Party Attack: Breaking Aggregation-Based Privacy in Federated Learning using Independent Component Analysis
• Authors: Sanjay Kariyappa, Chuan Guo, Kiwan Maeng, Wenjie Xiong, G. Edward Suh, Moinuddin K Qureshi, Hsien-Hsin S. Lee
• Abstract summary: Federated learning (FL) aims to perform privacy-preserving machine learning on distributed data held by multiple data owners. To this end, FL requires the data owners to perform training locally and share the gradient updates with the central server, which are then securely aggregated over multiple data owners. Although aggregation by itself does not provably offer privacy protection, prior work showed that it may suffice if the batch size is sufficiently large.
• Score: 26.233860960220483
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning (FL) aims to perform privacy-preserving machine learning on distributed data held by multiple data owners. To this end, FL requires the data owners to perform training locally and share the gradient updates (instead of the private inputs) with the central server, which are then securely aggregated over multiple data owners. Although aggregation by itself does not provably offer privacy protection, prior work showed that it may suffice if the batch size is sufficiently large. In this paper, we propose the Cocktail Party Attack (CPA) that, contrary to prior belief, is able to recover the private inputs from gradients aggregated over a very large batch size. CPA leverages the crucial insight that aggregate gradients from a fully connected layer is a linear combination of its inputs, which leads us to frame gradient inversion as a blind source separation (BSS) problem (informally called the cocktail party problem). We adapt independent component analysis (ICA)--a classic solution to the BSS problem--to recover private inputs for fully-connected and convolutional networks, and show that CPA significantly outperforms prior gradient inversion attacks, scales to ImageNet-sized inputs, and works on large batch sizes of up to 1024.

Related papers

• Subgraph Federated Learning via Spectral Methods [52.40322201034717]
  FedLap is a novel framework that captures inter-node dependencies while ensuring privacy and scalability.<n>We provide a formal analysis of the privacy of FedLap, demonstrating that it preserves privacy.
  arXiv  Detail & Related papers  (2025-10-29T16:22:32Z)
• DMM: Distributed Matrix Mechanism for Differentially-Private Federated Learning using Packed Secret Sharing [51.336015600778396]
  Federated Learning (FL) has gained lots of traction recently, both in industry and academia. In FL, a machine learning model is trained using data from various end-users arranged in committees across several rounds. Since such data can often be sensitive, a primary challenge in FL is providing privacy while still retaining utility of the model.
  arXiv  Detail & Related papers  (2024-10-21T16:25:14Z)
• Perfect Gradient Inversion in Federated Learning: A New Paradigm from the Hidden Subset Sum Problem [21.546869377126125]
  Federated Learning (FL) has emerged as a popular paradigm for collaborative learning among multiple parties. We formulate the input reconstruction problem using the gradient information shared in FL as the Hidden Subset Sum Problem. Our analysis provides insights into why empirical input reconstruction attacks degrade with larger batch sizes.
  arXiv  Detail & Related papers  (2024-09-21T23:01:33Z)
• Privacy-Preserving Split Learning with Vision Transformers using Patch-Wise Random and Noisy CutMix [38.370923655357366]
  In computer vision, the vision transformer (ViT) has increasingly superseded the convolutional neural network (CNN) for improved accuracy and robustness. Split learning (SL) emerges as a viable solution, leveraging server-side resources to train ViTs while utilizing private data from distributed devices. We propose a novel privacy-preserving SL framework that injects Gaussian noise into smashed data and mixes randomly chosen patches of smashed data across clients, coined DP-CutMixSL.
  arXiv  Detail & Related papers  (2024-08-02T06:24:39Z)
• Breaking Secure Aggregation: Label Leakage from Aggregated Gradients in Federated Learning [11.18348760596715]
  Federated Learning exhibits privacy vulnerabilities under gradient inversion attacks (GIAs) We propose a stealthy label inference attack to bypass Secure Aggregation (SA) and recover individual clients' private labels. Our attack achieves large-scale label recovery with 100% accuracy on various datasets and model architectures.
  arXiv  Detail & Related papers  (2024-06-22T04:42:18Z)
• Differentially Private Clustered Federated Learning [4.768272342753616]
  Federated learning (FL) often incorporates differential privacy (DP) to provide rigorous data privacy guarantees. Previous works attempted to address high structured data heterogeneity in vanilla FL settings through clustering clients (a.k.a clustered FL) We propose an algorithm for differentially private clustered FL, which is robust to the DP noise in the system and identifies the underlying clients' clusters correctly.
  arXiv  Detail & Related papers  (2024-05-29T17:03:31Z)
• Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks [72.51255282371805]
  We prove a privacy bound for the KL divergence between model distributions on worst-case neighboring datasets. We find that this KL privacy bound is largely determined by the expected squared gradient norm relative to model parameters during training.
  arXiv  Detail & Related papers  (2023-10-31T16:13:22Z)
• Understanding Deep Gradient Leakage via Inversion Influence Functions [53.1839233598743]
  Deep Gradient Leakage (DGL) is a highly effective attack that recovers private training images from gradient vectors. We propose a novel Inversion Influence Function (I$2$F) that establishes a closed-form connection between the recovered images and the private gradients. We empirically demonstrate that I$2$F effectively approximated the DGL generally on different model architectures, datasets, attack implementations, and perturbation-based defenses.
  arXiv  Detail & Related papers  (2023-09-22T17:26:24Z)
• Compressed Private Aggregation for Scalable and Robust Federated Learning over Massive Networks [34.29747990203208]
  Federated learning (FL) is an emerging paradigm that allows a central server to train machine learning models using remote users' data.<n>FL faces challenges in preserving the privacy of local datasets, its sensitivity to poisoning attacks by malicious users, and its communication overhead.<n>We present compressed private aggregation (CPA), that allows massive deployments to simultaneously communicate at extremely low bit rates.
  arXiv  Detail & Related papers  (2023-08-01T13:36:33Z)
• Is Vertical Logistic Regression Privacy-Preserving? A Comprehensive Privacy Analysis and Beyond [57.10914865054868]
  We consider vertical logistic regression (VLR) trained with mini-batch descent gradient. We provide a comprehensive and rigorous privacy analysis of VLR in a class of open-source Federated Learning frameworks.
  arXiv  Detail & Related papers  (2022-07-19T05:47:30Z)
• Mixed Differential Privacy in Computer Vision [133.68363478737058]
  AdaMix is an adaptive differentially private algorithm for training deep neural network classifiers using both private and public image data. A few-shot or even zero-shot learning baseline that ignores private data can outperform fine-tuning on a large private dataset.
  arXiv  Detail & Related papers  (2022-03-22T06:15:43Z)
• BEAS: Blockchain Enabled Asynchronous & Secure Federated Machine Learning [0.0]
  We present BEAS, the first blockchain-based framework for N-party Federated Learning. It provides strict privacy guarantees of training data using gradient pruning. Anomaly detection protocols are used to minimize the risk of data-poisoning attacks. We also define a novel protocol to prevent premature convergence in heterogeneous learning environments.
  arXiv  Detail & Related papers  (2022-02-06T17:11:14Z)
• Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy [67.4471689755097]
  This paper empirically demonstrates that the clipped FedAvg can perform surprisingly well even with substantial data heterogeneity. We provide the convergence analysis of a differential private (DP) FedAvg algorithm and highlight the relationship between clipping bias and the distribution of the clients' updates.
  arXiv  Detail & Related papers  (2021-06-25T14:47:19Z)
• WAFFLe: Weight Anonymized Factorization for Federated Learning [88.44939168851721]
  In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices. We propose Weight Anonymized Factorization for Federated Learning (WAFFLe), an approach that combines the Indian Buffet Process with a shared dictionary of weight factors for neural networks.
  arXiv  Detail & Related papers  (2020-08-13T04:26:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.