Agentic Evaluation of Copyright Law Compliance

• URL: http://arxiv.org/abs/2607.21799v2
• Date: Mon, 27 Jul 2026 09:56:47 GMT
• Title: Agentic Evaluation of Copyright Law Compliance
• Abstract summary: Copyright-Bench is a benchmark designed to evaluate large language model (LLM) agents' compliance with copyright law.<n>The evaluation introduces prompt variations that simulate different user preferences, as well as time pressure.<n>Comparing state-of-the-art LLM agents against a human baseline, we find that: (1) agents select copyrighted works despite the availability of public-domain alternatives; and (2) for open-weight models, violation rates increase in response to certain user preferences and simulated time pressure.
• Score: 4.4754237372329255
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM) agents increasingly perform commercial tasks that involve retrieving external content, such as images, and, where appropriate, reproducing that content. LLM agents should comply with the law, including copyright law. Presently, however, we lack adequate frameworks to assess whether they do so in practice. To that end, we introduce Copyright-Bench, a benchmark designed to evaluate LLM agents' compliance with copyright law. Copyright-Bench comprises realistic commercial tasks---website development, merchandise design, and pitch deck production---that involve agents selecting between public-domain content, the use of which is legal, and copyrighted content, the use of which is infringing in this setting. The evaluation introduces prompt variations that simulate different user preferences, as well as time pressure. Comparing state-of-the-art LLM agents against a human baseline, we find that: (1) agents select copyrighted works despite the availability of public-domain alternatives; and (2) for open-weight models, violation rates increase in response to certain user preferences and simulated time pressure.

Related papers

• Bridging the Copyright Gap: Do Large Vision-Language Models Recognize and Respect Copyrighted Content? [47.50752173848172]
  Large vision-language models (LVLMs) have achieved remarkable advancements in multimodal reasoning tasks.<n>Will LVLMs accurately recognize and comply with copyright regulations when encountering copyrighted content in the context?
  arXiv  Detail & Related papers  (2025-12-26T05:09:55Z)
• LAQuer: Localized Attribution Queries in Content-grounded Generation [69.60308443863606]
  Grounded text generation models often produce content that deviates from their source material, requiring user verification to ensure accuracy.<n>Existing attribution methods associate entire sentences with source documents, which can be overwhelming for users seeking to fact-check specific claims.<n>We introduce Localized Attribution Queries (LAQuer), a new task that localizes selected spans of generated output to their corresponding source spans, allowing fine-grained and user-directed attribution.
  arXiv  Detail & Related papers  (2025-06-01T21:46:23Z)
• CoTGuard: Using Chain-of-Thought Triggering for Copyright Protection in Multi-Agent LLM Systems [55.57181090183713]
  We introduce CoTGuard, a novel framework for copyright protection that leverages trigger-based detection within Chain-of-Thought reasoning.<n>Specifically, we can activate specific CoT segments and monitor intermediate reasoning steps for unauthorized content reproduction by embedding specific trigger queries into agent prompts.<n>This approach enables fine-grained, interpretable detection of copyright violations in collaborative agent scenarios.
  arXiv  Detail & Related papers  (2025-05-26T01:42:37Z)
• LegalAgentBench: Evaluating LLM Agents in Legal Domain [53.70993264644004]
  LegalAgentBench is a benchmark specifically designed to evaluate LLM Agents in the Chinese legal domain.<n>LegalAgentBench includes 17 corpora from real-world legal scenarios and provides 37 tools for interacting with external knowledge.
  arXiv  Detail & Related papers  (2024-12-23T04:02:46Z)
• Do LLMs Know to Respect Copyright Notice? [11.14140288980773]
  We investigate whether language models infringe upon copyrights when processing user input containing protected material. Our study offers a conservative evaluation of the extent to which language models may infringe upon copyrights. This research emphasizes the need for further investigation and the importance of ensuring LLMs respect copyright regulations.
  arXiv  Detail & Related papers  (2024-11-02T04:45:21Z)
• Measuring Copyright Risks of Large Language Model via Partial Information Probing [14.067687792633372]
  We explore the data sources used to train Large Language Models (LLMs) We input a portion of a copyrighted text into LLMs, prompt them to complete it, and then analyze the overlap between the generated content and the original copyrighted material. Our findings demonstrate that LLMs can indeed generate content highly overlapping with copyrighted materials based on these partial inputs.
  arXiv  Detail & Related papers  (2024-09-20T18:16:05Z)
• RLCP: A Reinforcement Learning-based Copyright Protection Method for Text-to-Image Diffusion Model [42.77851688874563]
  We propose a Reinforcement Learning-based Copyright Protection(RLCP) method for Text-to-Image Diffusion Model.<n>Our approach minimizes the generation of copyright-infringing content while maintaining the quality of the model-generated dataset.
  arXiv  Detail & Related papers  (2024-08-29T15:39:33Z)
• Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data? [62.72729485995075]
  We investigate the effectiveness of watermarking as a deterrent against the generation of copyrighted texts.<n>We find that watermarking adversely affects the success rate of Membership Inference Attacks (MIAs)<n>We propose an adaptive technique to improve the success rate of a recent MIA under watermarking.
  arXiv  Detail & Related papers  (2024-07-24T16:53:09Z)
• LLMs and Memorization: On Quality and Specificity of Copyright Compliance [0.0]
  Memorization in large language models (LLMs) is a growing concern. LLMs have been shown to easily reproduce parts of their training data, including copyrighted work. This is an important problem to solve, as it may violate existing copyright laws as well as the European AI Act.
  arXiv  Detail & Related papers  (2024-05-28T18:01:52Z)
• ©Plug-in Authorization for Human Content Copyright Protection in Text-to-Image Model [71.47762442337948]
  State-of-the-art models create high-quality content without crediting original creators.<n>We propose the copyright Plug-in Authorization framework, introducing three operations: addition, extraction, and combination.<n>Experiments in artist-style replication and cartoon IP recreation demonstrate copyright plug-ins' effectiveness.
  arXiv  Detail & Related papers  (2024-04-18T07:48:00Z)
• A Dataset and Benchmark for Copyright Infringement Unlearning from Text-to-Image Diffusion Models [52.49582606341111]
  Copyright law confers creators the exclusive rights to reproduce, distribute, and monetize their creative works. Recent progress in text-to-image generation has introduced formidable challenges to copyright enforcement. We introduce a novel pipeline that harmonizes CLIP, ChatGPT, and diffusion models to curate a dataset.
  arXiv  Detail & Related papers  (2024-01-04T11:14:01Z)
• Foundation Models and Fair Use [96.04664748698103]
  In the U.S. and other countries, copyrighted content may be used to build foundation models without incurring liability due to the fair use doctrine. In this work, we survey the potential risks of developing and deploying foundation models based on copyrighted content. We discuss technical mitigations that can help foundation models stay in line with fair use.
  arXiv  Detail & Related papers  (2023-03-28T03:58:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.