DWBench: Holistic Evaluation of Watermark for Dataset Copyright Auditing

• URL: http://arxiv.org/abs/2602.13541v1
• Date: Sat, 14 Feb 2026 01:09:19 GMT
• Title: DWBench: Holistic Evaluation of Watermark for Dataset Copyright Auditing
• Authors: Xiao Ren, Xinyi Yu, Linkang Du, Min Chen, Yuanchao Shu, Zhou Su, Yunjun Gao, Zhikun Zhang,
• Abstract summary: dataset watermark technique holds promise for auditing and verifying usage.<n>We develop DWBench, a unified benchmark and open-source toolkit for systematically evaluating image dataset watermark techniques.<n>We present the results of two new metrics: sample significance for fine-grained watermark distinguishability and verification success rate for dataset-level auditing.
• Score: 43.881484429055654
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The surging demand for large-scale datasets in deep learning has heightened the need for effective copyright protection, given the risks of unauthorized use to data owners. Although the dataset watermark technique holds promise for auditing and verifying usage, existing methods are hindered by inconsistent evaluations, which impede fair comparisons and assessments of real-world viability. To address this gap, we propose a two-layer taxonomy that categorizes methods by implementation (model-based vs. model-free injection; model-behavior vs. model-message verification), offering a structured framework for cross-task analysis. Then, we develop DWBench, a unified benchmark and open-source toolkit for systematically evaluating image dataset watermark techniques in classification and generation tasks. Using DWBench, we assess 25 representative methods under standardized conditions, perturbation-based robustness tests, multi-watermark coexistence, and multi-user interference. In addition to reporting the results of four commonly used metrics, we present the results of two new metrics: sample significance for fine-grained watermark distinguishability and verification success rate for dataset-level auditing, which enable accurate and reproducible benchmarking. Key findings reveal inherent trade-offs: no single method dominates all scenarios; classification and generation tasks require specialized approaches; and existing techniques exhibit instability at low watermark rates and in realistic multi-user settings, with elevated false positives or performance declines. We hope that DWBench can facilitate advances in watermark reliability and practicality, thus strengthening copyright safeguards in the face of widespread AI-driven data exploitation.

Related papers

• SEAL: Subspace-Anchored Watermarks for LLM Ownership [12.022506016268112]
  We propose SEAL, a subspace-anchored watermarking framework for large language models.<n> SEAL embeds multi-bit signatures directly into the model's latent representational space, supporting both white-box and black-box verification scenarios.<n>We conduct comprehensive experiments on multiple benchmark datasets and six prominent LLMs to demonstrate SEAL's superior effectiveness, fidelity, efficiency, and robustness.
  arXiv  Detail & Related papers  (2025-11-14T14:44:11Z)
• SWAP: Towards Copyright Auditing of Soft Prompts via Sequential Watermarking [58.475471437150674]
  We propose sequential watermarking for soft prompts (SWAP)<n>SWAP encodes watermarks through a specific order of defender-specified out-of-distribution classes.<n>Experiments on 11 datasets demonstrate SWAP's effectiveness, harmlessness, and robustness against potential adaptive attacks.
  arXiv  Detail & Related papers  (2025-11-05T13:48:48Z)
• SSCL-BW: Sample-Specific Clean-Label Backdoor Watermarking for Dataset Ownership Verification [8.045712223215542]
  This paper proposes a sample-specific clean-label backdoor watermarking (i.e., SSCL-BW)<n>By training a U-Net-based watermarked sample generator, this method generates unique watermarks for each sample.<n>Experiments on benchmark datasets demonstrate the effectiveness of the proposed method and its robustness against potential watermark removal attacks.
  arXiv  Detail & Related papers  (2025-10-30T12:13:53Z)
• CertDW: Towards Certified Dataset Ownership Verification via Conformal Prediction [48.82467166657901]
  We propose the first certified dataset watermark (i.e., CertDW) and CertDW-based certified dataset ownership verification method.<n>Inspired by conformal prediction, we introduce two statistical measures, including principal probability (PP) and watermark robustness (WR)<n>We prove there exists a provable lower bound between PP and WR, enabling ownership verification when a suspicious model's WR value significantly exceeds the PP values of benign models trained on watermark-free datasets.
  arXiv  Detail & Related papers  (2025-06-16T07:17:23Z)
• CBW: Towards Dataset Ownership Verification for Speaker Verification via Clustering-based Backdoor Watermarking [85.68235482145091]
  Large-scale speech datasets have become valuable intellectual property.<n>We propose a novel dataset ownership verification method.<n>Our approach introduces a clustering-based backdoor watermark (CBW)<n>We conduct extensive experiments on benchmark datasets, verifying the effectiveness and robustness of our method against potential adaptive attacks.
  arXiv  Detail & Related papers  (2025-03-02T02:02:57Z)
• Hide in Plain Sight: Clean-Label Backdoor for Auditing Membership Inference [16.893873979953593]
  We propose a novel clean-label backdoor-based approach for stealthy data auditing. Our approach employs an optimal trigger generated by a shadow model that mimics target model's behavior. The proposed method enables robust data auditing through blackbox access, achieving high attack success rates across diverse datasets.
  arXiv  Detail & Related papers  (2024-11-24T20:56:18Z)
• Thinking Racial Bias in Fair Forgery Detection: Models, Datasets and Evaluations [63.52709761339949]
  We first contribute a dedicated dataset called the Fair Forgery Detection (FairFD) dataset, where we prove the racial bias of public state-of-the-art (SOTA) methods.<n>We design novel metrics including Approach Averaged Metric and Utility Regularized Metric, which can avoid deceptive results.<n>We also present an effective and robust post-processing technique, Bias Pruning with Fair Activations (BPFA), which improves fairness without requiring retraining or weight updates.
  arXiv  Detail & Related papers  (2024-07-19T14:53:18Z)
• Model Stealing Attack against Graph Classification with Authenticity, Uncertainty and Diversity [80.16488817177182]
  GNNs are vulnerable to the model stealing attack, a nefarious endeavor geared towards duplicating the target model via query permissions. We introduce three model stealing attacks to adapt to different actual scenarios.
  arXiv  Detail & Related papers  (2023-12-18T05:42:31Z)
• Non-Transferable Learning: A New Approach for Model Verification and Authorization [7.686781778077341]
  There are two common protection methods: ownership verification and usage authorization. We propose Non-Transferable Learning (NTL), a novel approach that captures the exclusive data representation in the learned model. Our NTL-based authorization approach provides data-centric usage protection by significantly degrading the performance of usage on unauthorized data.
  arXiv  Detail & Related papers  (2021-06-13T04:57:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.