On the Effectiveness of Dataset Watermarking in Adversarial Settings

• URL: http://arxiv.org/abs/2202.12506v1
• Date: Fri, 25 Feb 2022 05:51:53 GMT
• Title: On the Effectiveness of Dataset Watermarking in Adversarial Settings
• Authors: Buse Gul Atli Tekgul, N. Asokan
• Abstract summary: We investigate a proposed data provenance method, radioactive data, to assess if it can be used to demonstrate ownership of (image) datasets used to train machine learning (ML) models. We show that radioactive data can effectively survive model extraction attacks, which raises the possibility that it can be used for ML model ownership verification robust against model extraction.
• Score: 14.095584034871658
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In a data-driven world, datasets constitute a significant economic value. Dataset owners who spend time and money to collect and curate the data are incentivized to ensure that their datasets are not used in ways that they did not authorize. When such misuse occurs, dataset owners need technical mechanisms for demonstrating their ownership of the dataset in question. Dataset watermarking provides one approach for ownership demonstration which can, in turn, deter unauthorized use. In this paper, we investigate a recently proposed data provenance method, radioactive data, to assess if it can be used to demonstrate ownership of (image) datasets used to train machine learning (ML) models. The original paper reported that radioactive data is effective in white-box settings. We show that while this is true for large datasets with many classes, it is not as effective for datasets where the number of classes is low $(\leq 30)$ or the number of samples per class is low $(\leq 500)$. We also show that, counter-intuitively, the black-box verification technique is effective for all datasets used in this paper, even when white-box verification is not. Given this observation, we show that the confidence in white-box verification can be improved by using watermarked samples directly during the verification process. We also highlight the need to assess the robustness of radioactive data if it were to be used for ownership demonstration since it is an adversarial setting unlike provenance identification. Compared to dataset watermarking, ML model watermarking has been explored more extensively in recent literature. However, most of the model watermarking techniques can be defeated via model extraction. We show that radioactive data can effectively survive model extraction attacks, which raises the possibility that it can be used for ML model ownership verification robust against model extraction.

Related papers

• Data Taggants: Dataset Ownership Verification via Harmless Targeted Data Poisoning [12.80649024603656]
  This paper introduces data taggants, a novel non-backdoor dataset ownership verification technique. We validate our approach through comprehensive and realistic experiments on ImageNet1k using ViT and ResNet models with state-of-the-art training recipes.
  arXiv  Detail & Related papers  (2024-10-09T12:49:23Z)
• PointNCBW: Towards Dataset Ownership Verification for Point Clouds via Negative Clean-label Backdoor Watermark [20.746346834429925]
  We propose a clean-label backdoor-based dataset watermark for point clouds that ensures both effectiveness and stealthiness. We perturb selected point clouds with non-target categories in both shape-wise and point-wise manners before inserting trigger patterns. As such, models trained on the watermarked dataset will have a distinctive yet stealthy backdoor behavior.
  arXiv  Detail & Related papers  (2024-08-10T09:31:58Z)
• TabularMark: Watermarking Tabular Datasets for Machine Learning [20.978995194849297]
  We propose a hypothesis testing-based watermarking scheme, TabularMark. Data noise partitioning is utilized for data perturbation during embedding. Experiments on real-world and synthetic datasets demonstrate the superiority of TabularMark in detectability, non-intrusiveness, and robustness.
  arXiv  Detail & Related papers  (2024-06-21T02:58:45Z)
• EnTruth: Enhancing the Traceability of Unauthorized Dataset Usage in Text-to-image Diffusion Models with Minimal and Robust Alterations [73.94175015918059]
  We introduce a novel approach, EnTruth, which Enhances Traceability of unauthorized dataset usage. By strategically incorporating the template memorization, EnTruth can trigger the specific behavior in unauthorized models as the evidence of infringement. Our method is the first to investigate the positive application of memorization and use it for copyright protection, which turns a curse into a blessing.
  arXiv  Detail & Related papers  (2024-06-20T02:02:44Z)
• FreqyWM: Frequency Watermarking for the New Data Economy [8.51675079658644]
  We present a novel technique for modulating the appearance frequency of a few tokens within a dataset for encoding an invisible watermark. We develop optimal as well as fast algorithms for creating and verifying such watermarks.
  arXiv  Detail & Related papers  (2023-12-27T12:17:59Z)
• Domain Watermark: Effective and Harmless Dataset Copyright Protection is Closed at Hand [96.26251471253823]
  backdoor-based dataset ownership verification (DOV) is currently the only feasible approach to protect the copyright of open-source datasets. We make watermarked models (trained on the protected dataset) correctly classify some hard' samples that will be misclassified by the benign model.
  arXiv  Detail & Related papers  (2023-10-09T11:23:05Z)
• Did You Train on My Dataset? Towards Public Dataset Protection with Clean-Label Backdoor Watermarking [54.40184736491652]
  We propose a backdoor-based watermarking approach that serves as a general framework for safeguarding public-available data. By inserting a small number of watermarking samples into the dataset, our approach enables the learning model to implicitly learn a secret function set by defenders. This hidden function can then be used as a watermark to track down third-party models that use the dataset illegally.
  arXiv  Detail & Related papers  (2023-03-20T21:54:30Z)
• Black-box Dataset Ownership Verification via Backdoor Watermarking [67.69308278379957]
  We formulate the protection of released datasets as verifying whether they are adopted for training a (suspicious) third-party model. We propose to embed external patterns via backdoor watermarking for the ownership verification to protect them. Specifically, we exploit poison-only backdoor attacks ($e.g.$, BadNets) for dataset watermarking and design a hypothesis-test-guided method for dataset verification.
  arXiv  Detail & Related papers  (2022-08-04T05:32:20Z)
• Hidden Biases in Unreliable News Detection Datasets [60.71991809782698]
  We show that selection bias during data collection leads to undesired artifacts in the datasets. We observed a significant drop (>10%) in accuracy for all models tested in a clean split with no train/test source overlap. We suggest future dataset creation include a simple model as a difficulty/bias probe and future model development use a clean non-overlapping site and date split.
  arXiv  Detail & Related papers  (2021-04-20T17:16:41Z)
• Open-sourced Dataset Protection via Backdoor Watermarking [87.15630326131901]
  We propose a emphbackdoor embedding based dataset watermarking method to protect an open-sourced image-classification dataset. We use a hypothesis test guided method for dataset verification based on the posterior probability generated by the suspicious third-party model.
  arXiv  Detail & Related papers  (2020-10-12T16:16:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.