Copyright-Protected Language Generation via Adaptive Model Fusion

• URL: http://arxiv.org/abs/2412.06619v1
• Date: Mon, 09 Dec 2024 16:13:17 GMT
• Title: Copyright-Protected Language Generation via Adaptive Model Fusion
• Authors: Javier Abad, Konstantin Donhauser, Francesco Pinto, Fanny Yang,
• Abstract summary: Copyright-Protecting Model Fusion (CP-Fuse) is a novel approach that combines models trained on disjoint sets of copyrighted material during inference. We show that CP-Fuse significantly reduces the reproduction of protected material without compromising the quality of text and code generation.
• Score: 15.48692649098646
• License:
• Abstract: The risk of language models reproducing copyrighted material from their training data has led to the development of various protective measures. Among these, inference-time strategies that impose constraints via post-processing have shown promise in addressing the complexities of copyright regulation. However, they often incur prohibitive computational costs or suffer from performance trade-offs. To overcome these limitations, we introduce Copyright-Protecting Model Fusion (CP-Fuse), a novel approach that combines models trained on disjoint sets of copyrighted material during inference. In particular, CP-Fuse adaptively aggregates the model outputs to minimize the reproduction of copyrighted content, adhering to a crucial balancing property that prevents the regurgitation of memorized data. Through extensive experiments, we show that CP-Fuse significantly reduces the reproduction of protected material without compromising the quality of text and code generation. Moreover, its post-hoc nature allows seamless integration with other protective measures, further enhancing copyright safeguards. Lastly, we show that CP-Fuse is robust against common techniques for extracting training data.

Related papers

• CopyrightShield: Spatial Similarity Guided Backdoor Defense against Copyright Infringement in Diffusion Models [61.06621533874629]
  diffusion model is a prime target for copyright infringement attacks. This paper provides an in-depth analysis of the spatial similarity of replication in diffusion model. We propose a novel defense method specifically targeting copyright infringement attacks.
  arXiv  Detail & Related papers  (2024-12-02T14:19:44Z)
• RLCP: A Reinforcement Learning-based Copyright Protection Method for Text-to-Image Diffusion Model [42.77851688874563]
  We propose a Reinforcement Learning-based Copyright Protection(RLCP) method for Text-to-Image Diffusion Model. Our approach minimizes the generation of copyright-infringing content while maintaining the quality of the model-generated dataset.
  arXiv  Detail & Related papers  (2024-08-29T15:39:33Z)
• Strong Copyright Protection for Language Models via Adaptive Model Fusion [15.48692649098646]
  Copyright-Protecting Fusion (CP-Fuse) is an algorithm that adaptively combines language models to minimize the reproduction of protected materials. Our results show that CP-Fuse significantly reduces the memorization of copyrighted content while maintaining high-quality text and code generation.
  arXiv  Detail & Related papers  (2024-07-29T15:32:30Z)
• Evaluating Copyright Takedown Methods for Language Models [100.38129820325497]
  Language models (LMs) derive their capabilities from extensive training on diverse data, including potentially copyrighted material. This paper introduces the first evaluation of the feasibility and side effects of copyright takedowns for LMs. We examine several strategies, including adding system prompts, decoding-time filtering interventions, and unlearning approaches.
  arXiv  Detail & Related papers  (2024-06-26T18:09:46Z)
• CPR: Retrieval Augmented Generation for Copyright Protection [101.15323302062562]
  We introduce CopyProtected generation with Retrieval (CPR), a new method for RAG with strong copyright protection guarantees. CPR allows to condition the output of diffusion models on a set of retrieved images. We prove that CPR satisfies Near Access Freeness (NAF) which bounds the amount of information an attacker may be able to extract from the generated images.
  arXiv  Detail & Related papers  (2024-03-27T18:09:55Z)
• Copyright Protection in Generative AI: A Technical Perspective [58.84343394349887]
  Generative AI has witnessed rapid advancement in recent years, expanding their capabilities to create synthesized content such as text, images, audio, and code. The high fidelity and authenticity of contents generated by these Deep Generative Models (DGMs) have sparked significant copyright concerns. This work delves into this issue by providing a comprehensive overview of copyright protection from a technical perspective.
  arXiv  Detail & Related papers  (2024-02-04T04:00:33Z)
• A Dataset and Benchmark for Copyright Infringement Unlearning from Text-to-Image Diffusion Models [52.49582606341111]
  Copyright law confers creators the exclusive rights to reproduce, distribute, and monetize their creative works. Recent progress in text-to-image generation has introduced formidable challenges to copyright enforcement. We introduce a novel pipeline that harmonizes CLIP, ChatGPT, and diffusion models to curate a dataset.
  arXiv  Detail & Related papers  (2024-01-04T11:14:01Z)
• Can Copyright be Reduced to Privacy? [23.639303165101385]
  We argue that while algorithmic stability may be perceived as a practical tool to detect copying, such copying does not necessarily constitute copyright infringement. If adopted as a standard for detecting an establishing copyright infringement, algorithmic stability may undermine the intended objectives of copyright law.
  arXiv  Detail & Related papers  (2023-05-24T07:22:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.