Quantifying Privacy Leakage in Split Inference via Fisher-Approximated Shannon Information Analysis

• URL: http://arxiv.org/abs/2504.10016v1
• Date: Mon, 14 Apr 2025 09:19:06 GMT
• Title: Quantifying Privacy Leakage in Split Inference via Fisher-Approximated Shannon Information Analysis
• Authors: Ruijun Deng, Zhihui Lu, Qiang Duan,
• Abstract summary: Split inference (SI) partitions deep neural networks into distributed sub-models, enabling privacy-preserving collaborative learning.<n>Despite extensive research on adversarial attack-defense games, a shortfall remains in the fundamental analysis of privacy risks.<n>This paper establishes a theoretical framework for privacy leakage quantification using information theory, defining it as the adversary's certainty and deriving both average-case and worst-case error bounds.
• Score: 1.4972864129791565
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Split inference (SI) partitions deep neural networks into distributed sub-models, enabling privacy-preserving collaborative learning. Nevertheless, it remains vulnerable to Data Reconstruction Attacks (DRAs), wherein adversaries exploit exposed smashed data to reconstruct raw inputs. Despite extensive research on adversarial attack-defense games, a shortfall remains in the fundamental analysis of privacy risks. This paper establishes a theoretical framework for privacy leakage quantification using information theory, defining it as the adversary's certainty and deriving both average-case and worst-case error bounds. We introduce Fisher-approximated Shannon information (FSInfo), a novel privacy metric utilizing Fisher Information (FI) for operational privacy leakage computation. We empirically show that our privacy metric correlates well with empirical attacks and investigate some of the factors that affect privacy leakage, namely the data distribution, model size, and overfitting.

Related papers

• Pseudo-Probability Unlearning: Towards Efficient and Privacy-Preserving Machine Unlearning [59.29849532966454]
  We propose PseudoProbability Unlearning (PPU), a novel method that enables models to forget data to adhere to privacy-preserving manner. Our method achieves over 20% improvements in forgetting error compared to the state-of-the-art.
  arXiv  Detail & Related papers  (2024-11-04T21:27:06Z)
• Bayes-Nash Generative Privacy Against Membership Inference Attacks [24.330984323956173]
  Membership inference attacks (MIAs) expose significant privacy risks by determining whether an individual's data is in a dataset.<n>We propose a game-theoretic framework that models privacy protection from MIA as a Bayesian game between a defender and an attacker.<n>We call the defender's data sharing policy thereby obtained Bayes-Nash Generative Privacy (BNGP)
  arXiv  Detail & Related papers  (2024-10-09T20:29:04Z)
• Differentially Private Data Release on Graphs: Inefficiencies and Unfairness [48.96399034594329]
  This paper characterizes the impact of Differential Privacy on bias and unfairness in the context of releasing information about networks. We consider a network release problem where the network structure is known to all, but the weights on edges must be released privately. Our work provides theoretical foundations and empirical evidence into the bias and unfairness arising due to privacy in these networked decision problems.
  arXiv  Detail & Related papers  (2024-08-08T08:37:37Z)
• Unveiling Privacy Vulnerabilities: Investigating the Role of Structure in Graph Data [17.11821761700748]
  This study advances the understanding and protection against privacy risks emanating from network structure. We develop a novel graph private attribute inference attack, which acts as a pivotal tool for evaluating the potential for privacy leakage through network structures. Our attack model poses a significant threat to user privacy, and our graph data publishing method successfully achieves the optimal privacy-utility trade-off.
  arXiv  Detail & Related papers  (2024-07-26T07:40:54Z)
• Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models [112.48136829374741]
  In this paper, we unveil a new vulnerability: the privacy backdoor attack. When a victim fine-tunes a backdoored model, their training data will be leaked at a significantly higher rate than if they had fine-tuned a typical model. Our findings highlight a critical privacy concern within the machine learning community and call for a reevaluation of safety protocols in the use of open-source pre-trained models.
  arXiv  Detail & Related papers  (2024-04-01T16:50:54Z)
• Visual Privacy Auditing with Diffusion Models [47.0700328585184]
  We introduce a reconstruction attack based on diffusion models (DMs) that only assumes adversary access to real-world image priors.<n>We find that (1) real-world data priors significantly influence reconstruction success, (2) current reconstruction bounds do not model the risk posed by data priors well, and DMs can serve as auditing tools for visualizing privacy leakage.
  arXiv  Detail & Related papers  (2024-03-12T12:18:55Z)
• Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks [72.51255282371805]
  We prove a privacy bound for the KL divergence between model distributions on worst-case neighboring datasets. We find that this KL privacy bound is largely determined by the expected squared gradient norm relative to model parameters during training.
  arXiv  Detail & Related papers  (2023-10-31T16:13:22Z)
• $\alpha$-Mutual Information: A Tunable Privacy Measure for Privacy Protection in Data Sharing [4.475091558538915]
  This paper adopts Arimoto's $alpha$-Mutual Information as a tunable privacy measure. We formulate a general distortion-based mechanism that manipulates the original data to offer privacy protection.
  arXiv  Detail & Related papers  (2023-10-27T16:26:14Z)
• Breaking the Communication-Privacy-Accuracy Tradeoff with $f$-Differential Privacy [51.11280118806893]
  We consider a federated data analytics problem in which a server coordinates the collaborative data analysis of multiple users with privacy concerns and limited communication capability. We study the local differential privacy guarantees of discrete-valued mechanisms with finite output space through the lens of $f$-differential privacy (DP) More specifically, we advance the existing literature by deriving tight $f$-DP guarantees for a variety of discrete-valued mechanisms.
  arXiv  Detail & Related papers  (2023-02-19T16:58:53Z)
• Federated Deep Learning with Bayesian Privacy [28.99404058773532]
  Federated learning (FL) aims to protect data privacy by cooperatively learning a model without sharing private data among users. Homomorphic encryption (HE) based methods provide secure privacy protections but suffer from extremely high computational and communication overheads. Deep learning with Differential Privacy (DP) was implemented as a practical learning algorithm at a manageable cost in complexity.
  arXiv  Detail & Related papers  (2021-09-27T12:48:40Z)
• Robustness Threats of Differential Privacy [70.818129585404]
  We experimentally demonstrate that networks, trained with differential privacy, in some settings might be even more vulnerable in comparison to non-private versions. We study how the main ingredients of differentially private neural networks training, such as gradient clipping and noise addition, affect the robustness of the model.
  arXiv  Detail & Related papers  (2020-12-14T18:59:24Z)
• Deep Directed Information-Based Learning for Privacy-Preserving Smart Meter Data Release [30.409342804445306]
  We study the problem in the context of time series data and smart meters (SMs) power consumption measurements. We introduce the Directed Information (DI) as a more meaningful measure of privacy in the considered setting. Our empirical studies on real-world data sets from SMs measurements in the worst-case scenario show the existing trade-offs between privacy and utility.
  arXiv  Detail & Related papers  (2020-11-20T13:41:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.