SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models

• URL: http://arxiv.org/abs/2604.17041v1
• Date: Sat, 18 Apr 2026 15:52:42 GMT
• Title: SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models
• Abstract summary: Existing ownership verification methods rely on semantically abnormal queries or out-of-distribution responses as fingerprints.<n>We propose SIF (Semantically In-Distribution Fingerprints), a non-intrusive ownership verification framework that requires no parameter modification.<n>SIF achieves strong stealthiness and robustness, providing a practical solution for LVLM copyright protection.
• Score: 18.815997579213317
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The public accessibility of large vision-language models (LVLMs) raises serious concerns about unauthorized model reuse and intellectual property infringement. Existing ownership verification methods often rely on semantically abnormal queries or out-of-distribution responses as fingerprints, which can be easily detected and removed by adversaries. We expose this vulnerability through a Semantic Divergence Attack (SDA), which identifies and filters fingerprint queries by measuring semantic divergence between a suspect model and a reference model, showing that existing fingerprints are not semantic-preserving and are therefore easy to detect and bypass. To address these limitations, we propose SIF (Semantically In-Distribution Fingerprints), a non-intrusive ownership verification framework that requires no parameter modification. SIF introduces Semantic-Aligned Fingerprint Distillation (SAFD), which transfers text watermarking signals into the visual modality to produce semantically coherent yet fingerprinted responses. In addition, Robust-Fingerprint Optimization (RFO) enhances robustness by simulating worst-case representation perturbations, making the fingerprints resilient to model modifications such as fine-tuning and quantization. Extensive experiments on LLaVA-1.5 and Qwen2.5-VL demonstrate that SIF achieves strong stealthiness and robustness, providing a practical solution for LVLM copyright protection. Code is available at https://github.com/UCF-ML-Research/SIF-VLM-Fingerprint

Related papers

• The Shape of Ownership: Verifying LLM Provenance through Semantic Structures [38.16125764360159]
  We introduce PROSE, replacing fixed query sets with a target semantical domain and brittle response keys with semantic structures as domain-conditioned response behavior.<n> PROSE constructs a private bank of domain-specific semantic templates, internalizes them through mixed fine-tuning on structurally verified and clean responses, and verifies ownership by detecting the designated structures in responses to held-out natural queries.
  arXiv  Detail & Related papers  (2026-09-02T13:04:51Z)
• Efficient, Robust, and Anti-Collusion Fingerprinting of Image Diffusion Models [99.66311024852651]
  Existing model fingerprinting methods lack robustness against collusion attacks.<n>We propose a robust fingerprinting method for T2I models with anti-collusion capabilities.<n>Our method achieves high fidelity and robustness across multiple T2I image generation and editing tasks, with fingerprint extraction accuracy exceeding 99.5%.
  arXiv  Detail & Related papers  (2026-06-11T07:12:13Z)
• Cert-LAS: Toward Certified Model Ownership Verification for Text-to-Image Diffusion Models via Layer-Adaptive Smoothing [75.56230265544042]
  Large-scale text-to-image (T2I) diffusion models have enabled unprecedented creative applications, but their unauthorized use has raised serious intellectual property concerns.<n>We propose Cert-LAS, the first certified MOV method for T2I models based on layer-adaptive smoothing.<n>Under certain conditions, our Cert-LAS can still achieve reliable verification even in the presence of malicious removal attacks.
  arXiv  Detail & Related papers  (2026-05-28T11:54:52Z)
• A Behavioral Fingerprint for Large Language Models: Provenance Tracking via Refusal Vectors [43.11304710234668]
  We introduce a novel fingerprinting framework that leverages the behavioral patterns induced by safety alignment.<n>In a large-scale identification task across 76 offspring models, our method achieves 100% accuracy in identifying the correct base model family.<n>We propose a theoretical framework to transform this private fingerprint into a publicly verifiable, privacy-preserving artifact.
  arXiv  Detail & Related papers  (2026-02-10T05:57:35Z)
• Antidistillation Fingerprinting [119.66677613290359]
  We introduce antidistillation fingerprinting (ADFP), a principled approach that aligns the fingerprinting objective with the student's learning dynamics.<n>ADFP achieves a significant improvement over state-of-the-art baselines, stronger detection confidence with minimal impact on utility, even when the student model's architecture is unknown.
  arXiv  Detail & Related papers  (2026-02-03T18:15:50Z)
• SWAP: Towards Copyright Auditing of Soft Prompts via Sequential Watermarking [58.475471437150674]
  We propose sequential watermarking for soft prompts (SWAP)<n>SWAP encodes watermarks through a specific order of defender-specified out-of-distribution classes.<n>Experiments on 11 datasets demonstrate SWAP's effectiveness, harmlessness, and robustness against potential adaptive attacks.
  arXiv  Detail & Related papers  (2025-11-05T13:48:48Z)
• Semantic Visual Anomaly Detection and Reasoning in AI-Generated Images [96.43608872116347]
  AnomReason is a large-scale benchmark with structured annotations as quadruple textbfAnomAgent<n>AnomReason and AnomAgent serve as a foundation for measuring and improving the semantic plausibility of AI-generated images.
  arXiv  Detail & Related papers  (2025-10-11T14:09:24Z)
• From Injection to Defense: Constructing Edit-Based Fingerprints for Large Language Models [28.393476667026523]
  We propose RFEdit, a knowledge-editing framework that embeds a rule-based multilingual natural language fingerprint (MNLF) by modifying a sparse subset of model weights.<n>RFEdit is protected by Fingerprint Subspace-aware Fine-Tuning (FSFT), which mitigates fingerprint degradation during legitimate fine-tuning.
  arXiv  Detail & Related papers  (2025-09-03T08:22:04Z)
• SoK: Large Language Model Copyright Auditing via Fingerprinting [69.14570598973195]
  We introduce a unified framework and formal taxonomy that categorizes existing methods into white-box and black-box approaches.<n>We propose LeaFBench, the first systematic benchmark for evaluating LLM fingerprinting under realistic deployment scenarios.
  arXiv  Detail & Related papers  (2025-08-27T12:56:57Z)
• ImF: Implicit Fingerprint for Large Language Models [14.580290415247385]
  We introduce a novel adversarial attack named Generation Revision Intervention (GRI) attack.<n>GRI exploits the semantic fragility of current fingerprinting methods, effectively erasing fingerprints.<n>We propose a novel model fingerprint paradigm called Implicit Fingerprints (ImF)
  arXiv  Detail & Related papers  (2025-03-25T05:47:34Z)
• CopyJudge: Automated Copyright Infringement Identification and Mitigation in Text-to-Image Diffusion Models [58.58208005178676]
  We propose CopyJudge, a novel automated infringement identification framework.<n>We employ an abstraction-filtration-comparison test framework to assess the likelihood of infringement.<n>We introduce a general LVLM-based mitigation strategy that automatically optimize infringing prompts.
  arXiv  Detail & Related papers  (2025-02-21T08:09:07Z)
• FIT-Print: Towards False-claim-resistant Model Ownership Verification via Targeted Fingerprint [22.398234847594242]
  Model fingerprinting is a widely adopted approach to safeguard the intellectual property rights of open-source models.<n>In this paper, we reveal that they are vulnerable to false claim attacks where adversaries falsely assert ownership of any third-party model.<n>Motivated by these findings, we propose a targeted fingerprinting paradigm (i.e., FIT-Print) to counteract false claim attacks.
  arXiv  Detail & Related papers  (2025-01-26T13:00:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.