Public Diffusion Models, Private Images: Key-Controlled Inversion for Conditional Reconstruction

• URL: http://arxiv.org/abs/2606.22988v1
• Date: Mon, 22 Jun 2026 08:07:07 GMT
• Title: Public Diffusion Models, Private Images: Key-Controlled Inversion for Conditional Reconstruction
• Abstract summary: We present a key-controlled inversion framework that turns the inherent error propagation of diffusion models into a security asset.<n>By injecting key-dependent noise into the inversion formula, we ensure that only a user with the correct key can reconstruct the original image.
• Score: 5.226880496392369
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models are often deployed in settings where model parameters are publicly accessible (e.g., open-source libraries or released checkpoints). This white-box scenario creates a serious security risk: any user who obtains an intermediate latent representation can invert the process to recover the original input image. Most prior work on access control for generative models assumes a black-box model (i.e., parameters are kept secret), typically under an honest-but-curious adversary. By contrast, we address the more challenging and realistic white-box setting where all parameters are public. We present a key-controlled inversion framework that turns the inherent error propagation of diffusion models, which exponentially amplifies small perturbations, into a security asset. By injecting key-dependent noise into the inversion formula, we ensure that only a user with the correct key can reconstruct the original image; any other key yields unrecognizable output. Theoretically, by leveraging existing error-propagation theory for diffusion models, we prove that the resulting ciphertext distribution is IND-CPA secure and derive that the adversary's advantage is exponentially small in a tunable security parameter, hence negligible for any probabilistic polynomial-time (PPT) adversary. Experimentally, we validate these security guarantees across several models and datasets and further demonstrate cross-model robustness, that the injected key noise does not amplify the performance drop caused by model discrepancies.

Related papers

• Fingerprinting Text-to-Image Diffusion Models via Collapsed Generation [38.697895774925136]
  Collapsed generation is an intrinsic, model-dependent property of the learned generation process.<n>We show collapsed generation fingerprints can distinguish different source models with low confusion.<n>These fingerprints remain verifiable in fine-tuned derivatives and under common and adaptive model- or query-level obfuscations.
  arXiv  Detail & Related papers  (2026-08-12T07:12:38Z)
• Perfectly-Private Analog Secure Aggregation in Federated Learning [51.61616734974475]
  In federated learning, multiple parties train models locally and share their parameters with a central server, which aggregates them to update a global model.<n>In this paper, a novel secure parameter aggregation method is proposed that employs the torus rather than a finite field.
  arXiv  Detail & Related papers  (2025-09-10T15:22:40Z)
• Embedding Hidden Adversarial Capabilities in Pre-Trained Diffusion Models [1.534667887016089]
  We introduce a new attack paradigm that embeds hidden adversarial capabilities directly into diffusion models via fine-tuning.<n>The resulting tampered model generates high-quality images indistinguishable from those of the original.<n>We demonstrate the effectiveness and stealthiness of our approach, uncovering a covert attack vector that raises new security concerns.
  arXiv  Detail & Related papers  (2025-04-05T12:51:36Z)
• TERD: A Unified Framework for Safeguarding Diffusion Models Against Backdoors [36.07978634674072]
  Diffusion models are vulnerable to backdoor attacks that compromise their integrity. We propose TERD, a backdoor defense framework that builds unified modeling for current attacks. TERD secures a 100% True Positive Rate (TPR) and True Negative Rate (TNR) across datasets of varying resolutions.
  arXiv  Detail & Related papers  (2024-09-09T03:02:16Z)
• Open-Set Deepfake Detection: A Parameter-Efficient Adaptation Method with Forgery Style Mixture [81.93945602120453]
  We introduce an approach that is both general and parameter-efficient for face forgery detection.<n>We design a forgery-style mixture formulation that augments the diversity of forgery source domains.<n>We show that the designed model achieves state-of-the-art generalizability with significantly reduced trainable parameters.
  arXiv  Detail & Related papers  (2024-08-23T01:53:36Z)
• Watch the Watcher! Backdoor Attacks on Security-Enhancing Diffusion Models [65.30406788716104]
  This work investigates the vulnerabilities of security-enhancing diffusion models. We demonstrate that these models are highly susceptible to DIFF2, a simple yet effective backdoor attack. Case studies show that DIFF2 can significantly reduce both post-purification and certified accuracy across benchmark datasets and models.
  arXiv  Detail & Related papers  (2024-06-14T02:39:43Z)
• Invisible Backdoor Attacks on Diffusion Models [22.08671395877427]
  Recent research has brought to light the vulnerability of diffusion models to backdoor attacks. We present an innovative framework designed to acquire invisible triggers, enhancing the stealthiness and resilience of inserted backdoors.
  arXiv  Detail & Related papers  (2024-06-02T17:43:19Z)
• Lazy Layers to Make Fine-Tuned Diffusion Models More Traceable [70.77600345240867]
  A novel arbitrary-in-arbitrary-out (AIAO) strategy makes watermarks resilient to fine-tuning-based removal. Unlike the existing methods of designing a backdoor for the input/output space of diffusion models, in our method, we propose to embed the backdoor into the feature space of sampled subpaths. Our empirical studies on the MS-COCO, AFHQ, LSUN, CUB-200, and DreamBooth datasets confirm the robustness of AIAO.
  arXiv  Detail & Related papers  (2024-05-01T12:03:39Z)
• Differentially Private Counterfactuals via Functional Mechanism [47.606474009932825]
  We propose a novel framework to generate differentially private counterfactual (DPC) without touching the deployed model or explanation set. In particular, we train an autoencoder with the functional mechanism to construct noisy class prototypes, and then derive the DPC from the latent prototypes.
  arXiv  Detail & Related papers  (2022-08-04T20:31:22Z)
• Turning a Curse into a Blessing: Enabling In-Distribution-Data-Free Backdoor Removal via Stabilized Model Inversion [27.294396320665594]
  We introduce a novel bi-level optimization-based framework for model inversion. We find that reconstructed samples from a pre-trained generator's latent space are backdoor-free, even when utilizing signals from a backdoored model.
  arXiv  Detail & Related papers  (2022-06-14T17:32:04Z)
• CC-Cert: A Probabilistic Approach to Certify General Robustness of Neural Networks [58.29502185344086]
  In safety-critical machine learning applications, it is crucial to defend models against adversarial attacks. It is important to provide provable guarantees for deep learning models against semantically meaningful input transformations. We propose a new universal probabilistic certification approach based on Chernoff-Cramer bounds.
  arXiv  Detail & Related papers  (2021-09-22T12:46:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.