Expressive variational quantum circuits provide inherent privacy in federated learning

• URL: http://arxiv.org/abs/2309.13002v2
• Date: Mon, 25 Sep 2023 16:11:56 GMT
• Title: Expressive variational quantum circuits provide inherent privacy in federated learning
• Authors: Niraj Kumar, Jamie Heredge, Changhao Li, Shaltiel Eloul, Shree Hari Sureshbabu, and Marco Pistoia
• Abstract summary: Federated learning has emerged as a viable solution to train machine learning models without the need to share data with the central aggregator. Standard neural network-based federated learning models have been shown to be susceptible to data leakage from the gradients shared with the server. We show that expressive maps lead to inherent privacy against gradient inversion attacks.
• Score: 2.3255115473995134
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning has emerged as a viable distributed solution to train machine learning models without the actual need to share data with the central aggregator. However, standard neural network-based federated learning models have been shown to be susceptible to data leakage from the gradients shared with the server. In this work, we introduce federated learning with variational quantum circuit model built using expressive encoding maps coupled with overparameterized ans\"atze. We show that expressive maps lead to inherent privacy against gradient inversion attacks, while overparameterization ensures model trainability. Our privacy framework centers on the complexity of solving the system of high-degree multivariate Chebyshev polynomials generated by the gradients of quantum circuit. We present compelling arguments highlighting the inherent difficulty in solving these equations, both in exact and approximate scenarios. Additionally, we delve into machine learning-based attack strategies and establish a direct connection between overparameterization in the original federated learning model and underparameterization in the attack model. Furthermore, we provide numerical scaling arguments showcasing that underparameterization of the expressive map in the attack model leads to the loss landscape being swamped with exponentially many spurious local minima points, thus making it extremely hard to realize a successful attack. This provides a strong claim, for the first time, that the nature of quantum machine learning models inherently helps prevent data leakage in federated learning.

Related papers

• Exponential Quantum Communication Advantage in Distributed Inference and Learning [19.827903766111987]
  We present a framework for distributed computation over a quantum network. We show that for models within this framework, inference and training using gradient descent can be performed with exponentially less communication. We also show that models in this class can encode highly nonlinear features of their inputs, and their expressivity increases exponentially with model depth.
  arXiv  Detail & Related papers  (2023-10-11T02:19:50Z)
• Phantom Embeddings: Using Embedding Space for Model Regularization in Deep Neural Networks [12.293294756969477]
  The strength of machine learning models stems from their ability to learn complex function approximations from data. The complex models tend to memorize the training data, which results in poor regularization performance on test data. We present a novel approach to regularize the models by leveraging the information-rich latent embeddings and their high intra-class correlation.
  arXiv  Detail & Related papers  (2023-04-14T17:15:54Z)
• Mixed Precision Quantization to Tackle Gradient Leakage Attacks in Federated Learning [1.7205106391379026]
  Federated Learning (FL) enables collaborative model building among a large number of participants without the need for explicit data sharing. This approach shows vulnerabilities when privacy inference attacks are applied to it. In particular, in the event of a gradient leakage attack, which has a higher success rate in retrieving sensitive data from the model gradients, FL models are at higher risk due to the presence of communication in their inherent architecture.
  arXiv  Detail & Related papers  (2022-10-22T04:24:32Z)
• Learning to Learn with Generative Models of Neural Network Checkpoints [71.06722933442956]
  We construct a dataset of neural network checkpoints and train a generative model on the parameters. We find that our approach successfully generates parameters for a wide range of loss prompts. We apply our method to different neural network architectures and tasks in supervised and reinforcement learning.
  arXiv  Detail & Related papers  (2022-09-26T17:59:58Z)
• ClusterQ: Semantic Feature Distribution Alignment for Data-Free Quantization [111.12063632743013]
  We propose a new and effective data-free quantization method termed ClusterQ. To obtain high inter-class separability of semantic features, we cluster and align the feature distribution statistics. We also incorporate the intra-class variance to solve class-wise mode collapse.
  arXiv  Detail & Related papers  (2022-04-30T06:58:56Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Investigating the Relationship Between Dropout Regularization and Model Complexity in Neural Networks [0.0]
  Dropout Regularization serves to reduce variance in Deep Learning models. We explore the relationship between the dropout rate and model complexity by training 2,000 neural networks. We build neural networks that predict the optimal dropout rate given the number of hidden units in each dense layer.
  arXiv  Detail & Related papers  (2021-08-14T23:49:33Z)
• Level-Set Curvature Neural Networks: A Hybrid Approach [0.0]
  We present a hybrid strategy based on deep learning to compute mean curvature in the level-set method. The proposed inference system combines a dictionary of improved regression models with standard numerical schemes to estimate curvature more accurately. Our findings confirm that machine learning is a promising venue for devising viable solutions to the level-set method's numerical shortcomings.
  arXiv  Detail & Related papers  (2021-04-07T06:51:52Z)
• UVeQFed: Universal Vector Quantization for Federated Learning [179.06583469293386]
  Federated learning (FL) is an emerging approach to train such learning models without requiring the users to share their possibly private labeled data. In FL, each user trains its copy of the learning model locally. The server then collects the individual updates and aggregates them into a global model. We show that combining universal vector quantization methods with FL yields a decentralized training system in which the compression of the trained models induces only a minimum distortion.
  arXiv  Detail & Related papers  (2020-06-05T07:10:22Z)
• Belief Propagation Reloaded: Learning BP-Layers for Labeling Problems [83.98774574197613]
  We take one of the simplest inference methods, a truncated max-product Belief propagation, and add what is necessary to make it a proper component of a deep learning model. This BP-Layer can be used as the final or an intermediate block in convolutional neural networks (CNNs) The model is applicable to a range of dense prediction problems, is well-trainable and provides parameter-efficient and robust solutions in stereo, optical flow and semantic segmentation.
  arXiv  Detail & Related papers  (2020-03-13T13:11:35Z)
• Kernel and Rich Regimes in Overparametrized Models [69.40899443842443]
  We show that gradient descent on overparametrized multilayer networks can induce rich implicit biases that are not RKHS norms. We also demonstrate this transition empirically for more complex matrix factorization models and multilayer non-linear networks.
  arXiv  Detail & Related papers  (2020-02-20T15:43:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.