Privacy-preserving machine learning with tensor networks

• URL: http://arxiv.org/abs/2202.12319v3
• Date: Tue, 23 Jul 2024 20:47:04 GMT
• Title: Privacy-preserving machine learning with tensor networks
• Authors: Alejandro Pozas-Kerstjens, Senaida Hernández-Santana, José Ramón Pareja Monturiol, Marco Castrillón López, Giannicola Scarpa, Carlos E. González-Guillén, David Pérez-García,
• Abstract summary: We show that tensor network architectures have especially prospective properties for privacy-preserving machine learning. First, we describe a new privacy vulnerability that is present in feedforward neural networks, illustrating it in synthetic and real-world datasets. We rigorously prove that such conditions are satisfied by tensor-network architectures.
• Score: 37.01494003138908
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Tensor networks, widely used for providing efficient representations of low-energy states of local quantum many-body systems, have been recently proposed as machine learning architectures which could present advantages with respect to traditional ones. In this work we show that tensor network architectures have especially prospective properties for privacy-preserving machine learning, which is important in tasks such as the processing of medical records. First, we describe a new privacy vulnerability that is present in feedforward neural networks, illustrating it in synthetic and real-world datasets. Then, we develop well-defined conditions to guarantee robustness to such vulnerability, which involve the characterization of models equivalent under gauge symmetry. We rigorously prove that such conditions are satisfied by tensor-network architectures. In doing so, we define a novel canonical form for matrix product states, which has a high degree of regularity and fixes the residual gauge that is left in the canonical forms based on singular value decompositions. We supplement the analytical findings with practical examples where matrix product states are trained on datasets of medical records, which show large reductions on the probability of an attacker extracting information about the training dataset from the model's parameters. Given the growing expertise in training tensor-network architectures, these results imply that one may not have to be forced to make a choice between accuracy in prediction and ensuring the privacy of the information processed.

Related papers

• Exponential Quantum Communication Advantage in Distributed Inference and Learning [19.827903766111987]
  We present a framework for distributed computation over a quantum network. We show that for models within this framework, inference and training using gradient descent can be performed with exponentially less communication. We also show that models in this class can encode highly nonlinear features of their inputs, and their expressivity increases exponentially with model depth.
  arXiv  Detail & Related papers  (2023-10-11T02:19:50Z)
• Robust Graph Representation Learning via Predictive Coding [46.22695915912123]
  Predictive coding is a message-passing framework initially developed to model information processing in the brain. In this work, we build models that rely on the message-passing rule of predictive coding. We show that the proposed models are comparable to standard ones in terms of performance in both inductive and transductive tasks.
  arXiv  Detail & Related papers  (2022-12-09T03:58:22Z)
• Mutual information estimation for graph convolutional neural networks [0.0]
  We present an architecture-agnostic method for tracking a network's internal representations during training, which are then used to create a mutual information plane. We compare how the inductive bias introduced in graph-based architectures changes the mutual information plane relative to a fully connected neural network.
  arXiv  Detail & Related papers  (2022-03-31T08:30:04Z)
• Quasi-orthogonality and intrinsic dimensions as measures of learning and generalisation [55.80128181112308]
  We show that dimensionality and quasi-orthogonality of neural networks' feature space may jointly serve as network's performance discriminants. Our findings suggest important relationships between the networks' final performance and properties of their randomly initialised feature spaces.
  arXiv  Detail & Related papers  (2022-03-30T21:47:32Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Leveraging the structure of dynamical systems for data-driven modeling [111.45324708884813]
  We consider the impact of the training set and its structure on the quality of the long-term prediction. We show how an informed design of the training set, based on invariants of the system and the structure of the underlying attractor, significantly improves the resulting models.
  arXiv  Detail & Related papers  (2021-12-15T20:09:20Z)
• Locally Sparse Networks for Interpretable Predictions [7.362415721170984]
  We propose a framework for training locally sparse neural networks where the local sparsity is learned via a sample-specific gating mechanism. The sample-specific sparsity is predicted via a textitgating network, which is trained in tandem with the textitprediction network. We demonstrate that our method outperforms state-of-the-art models when predicting the target function with far fewer features per instance.
  arXiv  Detail & Related papers  (2021-06-11T15:46:50Z)
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning [50.24174211654775]
  We present a novel contrastive self-supervised learning framework for anomaly detection on attributed networks. Our framework fully exploits the local information from network data by sampling a novel type of contrastive instance pair. A graph neural network-based contrastive learning model is proposed to learn informative embedding from high-dimensional attributes and local structure.
  arXiv  Detail & Related papers  (2021-02-27T03:17:20Z)
• Estimating informativeness of samples with Smooth Unique Information [108.25192785062367]
  We measure how much a sample informs the final weights and how much it informs the function computed by the weights. We give efficient approximations of these quantities using a linearized network. We apply these measures to several problems, such as dataset summarization.
  arXiv  Detail & Related papers  (2021-01-17T10:29:29Z)
• A probabilistic generative model for semi-supervised training of coarse-grained surrogates and enforcing physical constraints through virtual observables [3.8073142980733]
  This paper provides a flexible, probabilistic framework that accounts for physical structure and information both in the training objectives and in the surrogate model itself. We advocate a probabilistic model in which equalities that are available from the physics can be introduced as virtual observables and can provide additional information through the likelihood.
  arXiv  Detail & Related papers  (2020-06-02T17:14:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.