LoRA-Key: User-Centric LoRA Watermarking for Text-to-Image Diffusion Models

• URL: http://arxiv.org/abs/2605.29569v1
• Date: Thu, 28 May 2026 08:17:28 GMT
• Title: LoRA-Key: User-Centric LoRA Watermarking for Text-to-Image Diffusion Models
• Abstract summary: Low-Rank Adaptation (LoRA) has become a widely used mechanism for customizing text-to-image diffusion models.<n>LoRA-Key is a user-centric LoRA watermarking framework that treats copyright protection as a reusable ownership key.<n>We show that LoRA-Key provides lightweight plug-and-play copyright protection while preserving generation quality and style fidelity.
• Score: 21.453621643582434
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Low-Rank Adaptation (LoRA) has become a widely used mechanism for customizing text-to-image diffusion models, enabling lightweight modules that are shared, reused, and commercialized as independent assets. This LoRA-centric ecosystem shifts copyright protection from foundation models to distributed LoRA modules, which are easy to copy, redistribute, or reuse without authorization. Existing watermarking methods either protect the base diffusion model or require watermark-aware retraining for each target LoRA, limiting their practicality in open community settings. To address this limitation, we propose LoRA-Key, a user-centric LoRA watermarking framework that treats copyright protection as a reusable ownership key. LoRA-Key encapsulates a recoverable secret message into a standalone user-specific Watermark LoRA, which can be attached to different target LoRAs through training-free linear superposition without per-LoRA retraining or structural modification. To train such a reusable key, we first establish a latent watermark prior in the frozen VAE latent space for robust message embedding and recovery, and then optimize the Watermark LoRA with message-conditioned watermark supervision and semantic consistency constraints. We further introduce Gradient Orthogonal Projection (GOP) to suppress watermark updates that conflict with semantic-preserving directions, reducing interference with generation fidelity and downstream style adaptation. Extensive experiments show that LoRA-Key provides lightweight plug-and-play copyright protection while preserving generation quality and style fidelity, and maintains robust ownership verification under image-level distortions, downstream fine-tuning, and multi-LoRA composition.

Related papers

• When LoRA Betrays: Backdooring Text-to-Image Models by Masquerading as Benign Adapters [10.859491015719088]
  Low-Rank Adaptation (LoRA) has emerged as a leading technique for efficiently fine-tuning text-to-image diffusion models.<n>MasqLoRA is the first systematic attack framework that leverages an independent LoRA module as the attack vehicle.<n>MasqLoRA achieves a high attack success rate of 99.8%.
  arXiv  Detail & Related papers  (2026-02-25T14:56:51Z)
• AuthenLoRA: Entangling Stylization with Imperceptible Watermarks for Copyright-Secure LoRA Adapters [52.556959321030966]
  Low-Rank Adaptation (LoRA) offers an efficient paradigm for customizing diffusion models.<n>Existing watermarking techniques either target base models or verify LoRA modules themselves.<n>We propose AuthenLoRA, a unified watermarking framework that embeds imperceptible, traceable watermarks directly into the LoRA training process.
  arXiv  Detail & Related papers  (2025-11-26T09:48:11Z)
• MOLM: Mixture of LoRA Markers [28.772539229108286]
  We propose a general watermarking framework that formulates the encoding problem as key-dependent of the parameters of a generative model.<n>Within this framework, we introduce Mixture of LoRA Markers (MOLM), a routing-based instantiation in which binary keys activate lightweight LoRA adapters inside residual and attention blocks.
  arXiv  Detail & Related papers  (2025-09-30T21:27:14Z)
• StableGuard: Towards Unified Copyright Protection and Tamper Localization in Latent Diffusion Models [55.05404953041403]
  We propose a novel framework that seamlessly integrates a binary watermark into the diffusion generation process.<n>We show that StableGuard consistently outperforms state-of-the-art methods in image fidelity, watermark verification, and tampering localization.
  arXiv  Detail & Related papers  (2025-09-22T16:35:19Z)
• ZKLoRA: Efficient Zero-Knowledge Proofs for LoRA Verification [0.20482269513546458]
  Low-Rank Adaptation (LoRA) is a widely adopted method for customizing large-scale language models.<n>In distributed, untrusted training environments, an open source base model user may want to use LoRA weights created by an external contributor.<n>We present ZKLoRA, a zero-knowledge verification protocol that relies on succinct proofs and our novel Multi-Party Inference procedure.
  arXiv  Detail & Related papers  (2025-01-21T23:20:33Z)
• Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning [57.36978335727009]
  Low-Rank Adaptation (LoRA) offers an efficient way to fine-tune large language models (LLMs) In this paper, we propose a framework that adaptively retrieves and composes multiple LoRAs based on input prompts.
  arXiv  Detail & Related papers  (2024-06-24T05:24:41Z)
• AquaLoRA: Toward White-box Protection for Customized Stable Diffusion Models via Watermark LoRA [67.68750063537482]
  Diffusion models have achieved remarkable success in generating high-quality images. Recent works aim to let SD models output watermarked content for post-hoc forensics. We propose textttmethod as the first implementation under this scenario.
  arXiv  Detail & Related papers  (2024-05-18T01:25:47Z)
• LoRATK: LoRA Once, Backdoor Everywhere in the Share-and-Play Ecosystem [55.2986934528672]
  We study how backdoors can be injected into task-enhancing LoRAs.<n>We find that with a simple, efficient, yet specific recipe, a backdoor LoRA can be trained once and then seamlessly merged with multiple LoRAs.<n>Our work is among the first to study this new threat model of training-free distribution of downstream-capable-yet-backdoor-injected LoRAs.
  arXiv  Detail & Related papers  (2024-02-29T20:25:16Z)
• CA-LoRA: Adapting Existing LoRA for Compressed LLMs to Enable Efficient Multi-Tasking on Personal Devices [78.16679232748196]
  We introduce a Compression-Aware LoRA (CA-LoRA) framework to transfer Large Language Models (LLMs) to other tasks. Experiment results demonstrate that CA-LoRA outperforms the vanilla LoRA methods applied to a compressed LLM. The source code of CA-LoRA is available at https://github.com/thunlp/CA-LoRA.
  arXiv  Detail & Related papers  (2023-07-15T04:37:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.