TOFU: Towards Obfuscated Federated Updates by Encoding Weight Updates into Gradients from Proxy Data

• URL: http://arxiv.org/abs/2201.08494v1
• Date: Fri, 21 Jan 2022 00:25:42 GMT
• Title: TOFU: Towards Obfuscated Federated Updates by Encoding Weight Updates into Gradients from Proxy Data
• Authors: Isha Garg, Manish Nagaraj, Kaushik Roy
• Abstract summary: We propose TOFU, a novel algorithm which generates proxy data that encodes the weight updates for each client in its gradients. We show that TOFU enables learning with less than 1% and 7% accuracy drops on MNIST and on CIFAR-10 datasets. This enables us to learn to near-full accuracy in a federated setup, while being 4x and 6.6x more communication efficient than the standard Federated Averaging algorithm.
• Score: 7.489265323050362
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Advances in Federated Learning and an abundance of user data have enabled rich collaborative learning between multiple clients, without sharing user data. This is done via a central server that aggregates learning in the form of weight updates. However, this comes at the cost of repeated expensive communication between the clients and the server, and concerns about compromised user privacy. The inversion of gradients into the data that generated them is termed data leakage. Encryption techniques can be used to counter this leakage, but at added expense. To address these challenges of communication efficiency and privacy, we propose TOFU, a novel algorithm which generates proxy data that encodes the weight updates for each client in its gradients. Instead of weight updates, this proxy data is now shared. Since input data is far lower in dimensional complexity than weights, this encoding allows us to send much lesser data per communication round. Additionally, the proxy data resembles noise, and even perfect reconstruction from data leakage attacks would invert the decoded gradients into unrecognizable noise, enhancing privacy. We show that TOFU enables learning with less than 1% and 7% accuracy drops on MNIST and on CIFAR-10 datasets, respectively. This drop can be recovered via a few rounds of expensive encrypted gradient exchange. This enables us to learn to near-full accuracy in a federated setup, while being 4x and 6.6x more communication efficient than the standard Federated Averaging algorithm on MNIST and CIFAR-10, respectively.

Related papers

• RFLPA: A Robust Federated Learning Framework against Poisoning Attacks with Secure Aggregation [2.2667044928324747]
  Federated learning (FL) allows multiple devices to train a model collaboratively without sharing their data. Despite its benefits, FL is vulnerable to privacy leakage and poisoning attacks. We propose a robust federated learning framework against poisoning attacks (RFLPA) based on SecAgg protocol.
  arXiv  Detail & Related papers  (2024-05-24T03:31:10Z)
• LOKI: Large-scale Data Reconstruction Attack against Federated Learning through Model Manipulation [25.03733882637947]
  We introduce LOKI, an attack that overcomes previous limitations and also breaks the anonymity of aggregation. With FedAVG and aggregation across 100 clients, prior work can leak less than 1% of images on MNIST, CIFAR-100, and Tiny ImageNet. Using only a single training round, LOKI is able to leak 76-86% of all data samples.
  arXiv  Detail & Related papers  (2023-03-21T23:29:35Z)
• Suppressing Poisoning Attacks on Federated Learning for Medical Imaging [4.433842217026879]
  We propose a robust aggregation rule called Distance-based Outlier Suppression (DOS) that is resilient to byzantine failures. The proposed method computes the distance between local parameter updates of different clients and obtains an outlier score for each client. The resulting outlier scores are converted into normalized weights using a softmax function, and a weighted average of the local parameters is used for updating the global model.
  arXiv  Detail & Related papers  (2022-07-15T00:43:34Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Do Gradient Inversion Attacks Make Federated Learning Unsafe? [70.0231254112197]
  Federated learning (FL) allows the collaborative training of AI models without needing to share raw data. Recent works on the inversion of deep neural networks from model gradients raised concerns about the security of FL in preventing the leakage of training data. In this work, we show that these attacks presented in the literature are impractical in real FL use-cases and provide a new baseline attack.
  arXiv  Detail & Related papers  (2022-02-14T18:33:12Z)
• BEAS: Blockchain Enabled Asynchronous & Secure Federated Machine Learning [0.0]
  We present BEAS, the first blockchain-based framework for N-party Federated Learning. It provides strict privacy guarantees of training data using gradient pruning. Anomaly detection protocols are used to minimize the risk of data-poisoning attacks. We also define a novel protocol to prevent premature convergence in heterogeneous learning environments.
  arXiv  Detail & Related papers  (2022-02-06T17:11:14Z)
• When the Curious Abandon Honesty: Federated Learning Is Not Private [36.95590214441999]
  In federated learning (FL), data does not leave personal devices when they are jointly training a machine learning model. We show a novel data reconstruction attack which allows an active and dishonest central party to efficiently extract user data from the received gradients.
  arXiv  Detail & Related papers  (2021-12-06T10:37:03Z)
• CAFE: Catastrophic Data Leakage in Vertical Federated Learning [65.56360219908142]
  Recent studies show that private training data can be leaked through the gradients sharing mechanism deployed in distributed machine learning systems. We propose an advanced data leakage attack with theoretical justification to efficiently recover batch data from the shared aggregated gradients.
  arXiv  Detail & Related papers  (2021-10-26T23:22:58Z)
• Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models [56.0250919557652]
  Federated learning has quickly gained popularity with its promises of increased user privacy and efficiency. Previous attacks on user privacy have been limited in scope and do not scale to gradient updates aggregated over even a handful of data points. We introduce a new threat model based on minimal but malicious modifications of the shared model architecture.
  arXiv  Detail & Related papers  (2021-10-25T15:52:06Z)
• WAFFLe: Weight Anonymized Factorization for Federated Learning [88.44939168851721]
  In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices. We propose Weight Anonymized Factorization for Federated Learning (WAFFLe), an approach that combines the Indian Buffet Process with a shared dictionary of weight factors for neural networks.
  arXiv  Detail & Related papers  (2020-08-13T04:26:31Z)
• Faster Secure Data Mining via Distributed Homomorphic Encryption [108.77460689459247]
  Homomorphic Encryption (HE) is receiving more and more attention recently for its capability to do computations over the encrypted field. We propose a novel general distributed HE-based data mining framework towards one step of solving the scaling problem. We verify the efficiency and effectiveness of our new framework by testing over various data mining algorithms and benchmark data-sets.
  arXiv  Detail & Related papers  (2020-06-17T18:14:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.