Strong Copyright Protection for Language Models via Adaptive Model Fusion

• URL: http://arxiv.org/abs/2407.20105v1
• Date: Mon, 29 Jul 2024 15:32:30 GMT
• Title: Strong Copyright Protection for Language Models via Adaptive Model Fusion
• Authors: Javier Abad, Konstantin Donhauser, Francesco Pinto, Fanny Yang,
• Abstract summary: Copyright-Protecting Fusion (CP-Fuse) is an algorithm that adaptively combines language models to minimize the reproduction of protected materials. Our results show that CP-Fuse significantly reduces the memorization of copyrighted content while maintaining high-quality text and code generation.
• Score: 15.48692649098646
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The risk of language models unintentionally reproducing copyrighted material from their training data has led to the development of various protective measures. In this paper, we propose model fusion as an effective solution to safeguard against copyright infringement. In particular, we introduce Copyright-Protecting Fusion (CP-Fuse), an algorithm that adaptively combines language models to minimize the reproduction of protected materials. CP-Fuse is inspired by the recently proposed Near-Access Free (NAF) framework and additionally incorporates a desirable balancing property that we demonstrate prevents the reproduction of memorized training data. Our results show that CP-Fuse significantly reduces the memorization of copyrighted content while maintaining high-quality text and code generation. Furthermore, we demonstrate how CP-Fuse can be integrated with other techniques for enhanced protection.

Related papers

• RLCP: A Reinforcement Learning-based Copyright Protection Method for Text-to-Image Diffusion Model [42.77851688874563]
  We propose a Reinforcement Learning-based Copyright Protection(RLCP) method for Text-to-Image Diffusion Model. Our approach minimizes the generation of copyright-infringing content while maintaining the quality of the model-generated dataset.
  arXiv  Detail & Related papers  (2024-08-29T15:39:33Z)
• Evaluating Copyright Takedown Methods for Language Models [100.38129820325497]
  Language models (LMs) derive their capabilities from extensive training on diverse data, including potentially copyrighted material. This paper introduces the first evaluation of the feasibility and side effects of copyright takedowns for LMs. We examine several strategies, including adding system prompts, decoding-time filtering interventions, and unlearning approaches.
  arXiv  Detail & Related papers  (2024-06-26T18:09:46Z)
• CPR: Retrieval Augmented Generation for Copyright Protection [101.15323302062562]
  We introduce CopyProtected generation with Retrieval (CPR), a new method for RAG with strong copyright protection guarantees. CPR allows to condition the output of diffusion models on a set of retrieved images. We prove that CPR satisfies Near Access Freeness (NAF) which bounds the amount of information an attacker may be able to extract from the generated images.
  arXiv  Detail & Related papers  (2024-03-27T18:09:55Z)
• JAMDEC: Unsupervised Authorship Obfuscation using Constrained Decoding over Small Language Models [53.83273575102087]
  We propose an unsupervised inference-time approach to authorship obfuscation. We introduce JAMDEC, a user-controlled, inference-time algorithm for authorship obfuscation. Our approach builds on small language models such as GPT2-XL in order to help avoid disclosing the original content to proprietary LLM's APIs.
  arXiv  Detail & Related papers  (2024-02-13T19:54:29Z)
• A Dataset and Benchmark for Copyright Infringement Unlearning from Text-to-Image Diffusion Models [52.49582606341111]
  Copyright law confers creators the exclusive rights to reproduce, distribute, and monetize their creative works. Recent progress in text-to-image generation has introduced formidable challenges to copyright enforcement. We introduce a novel pipeline that harmonizes CLIP, ChatGPT, and diffusion models to curate a dataset.
  arXiv  Detail & Related papers  (2024-01-04T11:14:01Z)
• EncryIP: A Practical Encryption-Based Framework for Model Intellectual Property Protection [17.655627250882805]
  This paper introduces a practical encryption-based framework called textitEncryIP. It seamlessly integrates a public-key encryption scheme into the model learning process. It demonstrates superior effectiveness in both training protected models and efficiently detecting the unauthorized spread of ML models.
  arXiv  Detail & Related papers  (2023-12-19T11:11:03Z)
• VA3: Virtually Assured Amplification Attack on Probabilistic Copyright Protection for Text-to-Image Generative Models [27.77911368516792]
  We introduce Virtually Assured Amplification Attack (VA3), a novel online attack framework. VA3 amplifies the probability of generating infringing content on the sustained interactions with generative models. These findings highlight the potential risk of implementing probabilistic copyright protection in practical applications of text-to-image generative models.
  arXiv  Detail & Related papers  (2023-11-29T12:10:00Z)
• Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark [58.60940048748815]
  Companies have begun to offer Embedding as a Service (E) based on large language models (LLMs) E is vulnerable to model extraction attacks, which can cause significant losses for the owners of LLMs. We propose an Embedding Watermark method called EmbMarker that implants backdoors on embeddings.
  arXiv  Detail & Related papers  (2023-05-17T08:28:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.