Tracking the Copyright of Large Vision-Language Models through Parameter Learning Adversarial Images

• URL: http://arxiv.org/abs/2502.16593v1
• Date: Sun, 23 Feb 2025 14:49:34 GMT
• Title: Tracking the Copyright of Large Vision-Language Models through Parameter Learning Adversarial Images
• Authors: Yubo Wang, Jianting Tang, Chaohu Liu, Linli Xu,
• Abstract summary: Large vision-language models (LVLMs) have demonstrated remarkable image understanding and dialogue capabilities.<n>Their widespread availability raises concerns about unauthorized usage and copyright infringement.<n>We propose a novel method called Learning Attack (PLA) for tracking the copyright of LVLMs without modifying the original model.
• Score: 9.351260848685229
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large vision-language models (LVLMs) have demonstrated remarkable image understanding and dialogue capabilities, allowing them to handle a variety of visual question answering tasks. However, their widespread availability raises concerns about unauthorized usage and copyright infringement, where users or individuals can develop their own LVLMs by fine-tuning published models. In this paper, we propose a novel method called Parameter Learning Attack (PLA) for tracking the copyright of LVLMs without modifying the original model. Specifically, we construct adversarial images through targeted attacks against the original model, enabling it to generate specific outputs. To ensure these attacks remain effective on potential fine-tuned models to trigger copyright tracking, we allow the original model to learn the trigger images by updating parameters in the opposite direction during the adversarial attack process. Notably, the proposed method can be applied after the release of the original model, thus not affecting the model's performance and behavior. To simulate real-world applications, we fine-tune the original model using various strategies across diverse datasets, creating a range of models for copyright verification. Extensive experiments demonstrate that our method can more effectively identify the original copyright of fine-tuned models compared to baseline methods. Therefore, this work provides a powerful tool for tracking copyrights and detecting unlicensed usage of LVLMs.

Related papers

• CopyJudge: Automated Copyright Infringement Identification and Mitigation in Text-to-Image Diffusion Models [58.58208005178676]
  We propose CopyJudge, an automated copyright infringement identification framework.<n>We employ an abstraction-filtration-comparison test framework with multi-LVLM debate to assess the likelihood of infringement.<n>Based on the judgments, we introduce a general LVLM-based mitigation strategy.
  arXiv  Detail & Related papers  (2025-02-21T08:09:07Z)
• SleeperMark: Towards Robust Watermark against Fine-Tuning Text-to-image Diffusion Models [77.80595722480074]
  SleeperMark is a framework designed to embed resilient watermarks into T2I diffusion models. It guides the model to disentangle the watermark information from the semantic concepts it learns. Our experiments demonstrate the effectiveness of SleeperMark across various types of diffusion models.
  arXiv  Detail & Related papers  (2024-12-06T08:44:18Z)
• RLCP: A Reinforcement Learning-based Copyright Protection Method for Text-to-Image Diffusion Model [42.77851688874563]
  We propose a Reinforcement Learning-based Copyright Protection(RLCP) method for Text-to-Image Diffusion Model.<n>Our approach minimizes the generation of copyright-infringing content while maintaining the quality of the model-generated dataset.
  arXiv  Detail & Related papers  (2024-08-29T15:39:33Z)
• Evaluating Copyright Takedown Methods for Language Models [100.38129820325497]
  Language models (LMs) derive their capabilities from extensive training on diverse data, including potentially copyrighted material. This paper introduces the first evaluation of the feasibility and side effects of copyright takedowns for LMs. We examine several strategies, including adding system prompts, decoding-time filtering interventions, and unlearning approaches.
  arXiv  Detail & Related papers  (2024-06-26T18:09:46Z)
• ProFLingo: A Fingerprinting-based Intellectual Property Protection Scheme for Large Language Models [18.46904928949022]
  We propose ProFLingo, a black-box fingerprinting-based IP protection scheme for large language models (LLMs) ProFLingo generates queries that elicit specific responses from an original model, thereby establishing unique fingerprints. Our scheme assesses the effectiveness of these queries on a suspect model to determine whether it has been derived from the original model.
  arXiv  Detail & Related papers  (2024-05-03T20:00:40Z)
• CGI-DM: Digital Copyright Authentication for Diffusion Models via Contrasting Gradient Inversion [26.70115339710056]
  Contrasting Gradient Inversion for Diffusion Models (CGI-DM) is a novel method featuring vivid visual representations for digital copyright authentication. We formulate the differences as KL divergence between latent variables of the two models when given the same input image. Experiments on the WikiArt and Dreambooth datasets demonstrate the high accuracy of CGI-DM in digital copyright authentication.
  arXiv  Detail & Related papers  (2024-03-17T10:06:38Z)
• Regeneration Based Training-free Attribution of Fake Images Generated by Text-to-Image Generative Models [39.33821502730661]
  We present a training-free method to attribute fake images generated by text-to-image models to their source models. By calculating and ranking the similarity of the test image and the candidate images, we can determine the source of the image.
  arXiv  Detail & Related papers  (2024-03-03T11:55:49Z)
• A Dataset and Benchmark for Copyright Infringement Unlearning from Text-to-Image Diffusion Models [52.49582606341111]
  Copyright law confers creators the exclusive rights to reproduce, distribute, and monetize their creative works. Recent progress in text-to-image generation has introduced formidable challenges to copyright enforcement. We introduce a novel pipeline that harmonizes CLIP, ChatGPT, and diffusion models to curate a dataset.
  arXiv  Detail & Related papers  (2024-01-04T11:14:01Z)
• DIAGNOSIS: Detecting Unauthorized Data Usages in Text-to-image Diffusion Models [79.71665540122498]
  We propose a method for detecting unauthorized data usage by planting the injected content into the protected dataset. Specifically, we modify the protected images by adding unique contents on these images using stealthy image warping functions. By analyzing whether the model has memorized the injected content, we can detect models that had illegally utilized the unauthorized data.
  arXiv  Detail & Related papers  (2023-07-06T16:27:39Z)
• MOVE: Effective and Harmless Ownership Verification via Embedded External Features [104.97541464349581]
  We propose an effective and harmless model ownership verification (MOVE) to defend against different types of model stealing simultaneously. We conduct the ownership verification by verifying whether a suspicious model contains the knowledge of defender-specified external features. We then train a meta-classifier to determine whether a model is stolen from the victim.
  arXiv  Detail & Related papers  (2022-08-04T02:22:29Z)
• DynaMarks: Defending Against Deep Learning Model Extraction Using Dynamic Watermarking [3.282282297279473]
  The functionality of a deep learning (DL) model can be stolen via model extraction. We propose a novel watermarking technique called DynaMarks to protect the intellectual property (IP) of DL models.
  arXiv  Detail & Related papers  (2022-07-27T06:49:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.