Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks

• URL: http://arxiv.org/abs/2305.10160v2
• Date: Wed, 18 Oct 2023 13:17:13 GMT
• Title: Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks
• Authors: Alon Jacovi, Avi Caciularu, Omer Goldman, Yoav Goldberg
• Abstract summary: Data contamination has become prevalent and challenging with the rise of models pretrained on large automatically-crawled corpora. For closed models, the training data becomes a trade secret, and even for open models, it is not trivial to detect contamination. We propose three strategies that can make a difference: (1) Test data made public should be encrypted with a public key and licensed to disallow derivative distribution; (2) demand training exclusion controls from closed API holders, and protect your test data by refusing to evaluate without them; and (3) avoid data which appears with its solution on the internet, and release the web-page context of internet-derived
• Score: 70.39633252935445
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data contamination has become prevalent and challenging with the rise of models pretrained on large automatically-crawled corpora. For closed models, the training data becomes a trade secret, and even for open models, it is not trivial to detect contamination. Strategies such as leaderboards with hidden answers, or using test data which is guaranteed to be unseen, are expensive and become fragile with time. Assuming that all relevant actors value clean test data and will cooperate to mitigate data contamination, what can be done? We propose three strategies that can make a difference: (1) Test data made public should be encrypted with a public key and licensed to disallow derivative distribution; (2) demand training exclusion controls from closed API holders, and protect your test data by refusing to evaluate without them; (3) avoid data which appears with its solution on the internet, and release the web-page context of internet-derived data along with the data. These strategies are practical and can be effective in preventing data contamination.

Related papers

• Data Taggants: Dataset Ownership Verification via Harmless Targeted Data Poisoning [12.80649024603656]
  This paper introduces data taggants, a novel non-backdoor dataset ownership verification technique. We validate our approach through comprehensive and realistic experiments on ImageNet1k using ViT and ResNet models with state-of-the-art training recipes.
  arXiv  Detail & Related papers  (2024-10-09T12:49:23Z)
• Training on the Benchmark Is Not All You Need [52.01920740114261]
  We propose a simple and effective data leakage detection method based on the contents of multiple-choice options. Our method is able to work under black-box conditions without access to model training data or weights. We evaluate the degree of data leakage of 31 mainstream open-source LLMs on four benchmark datasets.
  arXiv  Detail & Related papers  (2024-09-03T11:09:44Z)
• Footprints of Data in a Classifier Model: The Privacy Issues and Their Mitigation through Data Obfuscation [0.9208007322096533]
  embedding of footprints of training data in a prediction model is one such facet. difference in performance quality in test and training data causes passive identification of data that have trained the model. This research focuses on addressing the vulnerability arising from the data footprints.
  arXiv  Detail & Related papers  (2024-07-02T13:56:37Z)
• DataCook: Crafting Anti-Adversarial Examples for Healthcare Data Copyright Protection [47.91906879320081]
  DataCook operates by "cooking" the raw data before distribution, enabling the development of models that perform normally on this processed data. During the deployment phase, the original test data must be also "cooked" through DataCook to ensure normal model performance. The mechanism behind DataCook is by crafting anti-adversarial examples (AntiAdv), which are designed to enhance model confidence.
  arXiv  Detail & Related papers  (2024-03-26T14:44:51Z)
• Can We Trust the Unlabeled Target Data? Towards Backdoor Attack and Defense on Model Adaptation [120.42853706967188]
  We explore the potential backdoor attacks on model adaptation launched by well-designed poisoning target data. We propose a plug-and-play method named MixAdapt, combining it with existing adaptation algorithms.
  arXiv  Detail & Related papers  (2024-01-11T16:42:10Z)
• Mendata: A Framework to Purify Manipulated Training Data [12.406255198638064]
  We propose Mendata, a framework to purify manipulated training data. Mendata perturbs the training inputs so that they retain their utility but are distributed similarly to the reference data. We demonstrate the effectiveness of Mendata by applying it to defeat state-of-the-art data poisoning and data tracing techniques.
  arXiv  Detail & Related papers  (2023-12-03T04:40:08Z)
• Membership Inference Attacks against Synthetic Data through Overfitting Detection [84.02632160692995]
  We argue for a realistic MIA setting that assumes the attacker has some knowledge of the underlying data distribution. We propose DOMIAS, a density-based MIA model that aims to infer membership by targeting local overfitting of the generative model.
  arXiv  Detail & Related papers  (2023-02-24T11:27:39Z)
• Try to Avoid Attacks: A Federated Data Sanitization Defense for Healthcare IoMT Systems [4.024567343465081]
  The distribution of IoMT has the risk of protection from data poisoning attacks. Poisoned data can be fabricated by falsifying medical data. This paper introduces a Federated Data Sanitization Defense, a novel approach to protect the system from data poisoning attacks.
  arXiv  Detail & Related papers  (2022-11-03T05:21:39Z)
• Black-box Dataset Ownership Verification via Backdoor Watermarking [67.69308278379957]
  We formulate the protection of released datasets as verifying whether they are adopted for training a (suspicious) third-party model. We propose to embed external patterns via backdoor watermarking for the ownership verification to protect them. Specifically, we exploit poison-only backdoor attacks ($e.g.$, BadNets) for dataset watermarking and design a hypothesis-test-guided method for dataset verification.
  arXiv  Detail & Related papers  (2022-08-04T05:32:20Z)
• DAD: Data-free Adversarial Defense at Test Time [21.741026088202126]
  Deep models are highly susceptible to adversarial attacks. Privacy has become an important concern, restricting access to only trained models but not the training data. We propose a completely novel problem of 'test-time adversarial defense in absence of training data and even their statistics'
  arXiv  Detail & Related papers  (2022-04-04T15:16:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.