Detecting and Filtering Unsafe Training Data via Data Attribution

• URL: http://arxiv.org/abs/2502.11411v1
• Date: Mon, 17 Feb 2025 03:50:58 GMT
• Title: Detecting and Filtering Unsafe Training Data via Data Attribution
• Authors: Yijun Pan, Taiwei Shi, Jieyu Zhao, Jiaqi W. Ma,
• Abstract summary: Large language models (LLMs) are vulnerable to unsafe training data. We propose DABUF, leveraging data attribution to detect and filter unsafe training data. We evaluate the performance on two different tasks: in filtering jailbreaking training data and in identifying and mitigating gender bias.
• Score: 10.111622301509362
• License:
• Abstract: Large language models (LLMs) are vulnerable to unsafe training data that even small amounts of unsafe data can lead to harmful model behaviors. Detecting and filtering such unsafe training data is essential for trustworthy model development. Current state-of-the-art (SOTA) approaches typically rely on training moderation classifiers which requires significant computational overhead and are limited to predefined taxonomies, making them less adaptable to evolving safety concerns. Moreover, these classifiers lack insight into the training process, limiting their effectiveness in filtering unsafe data. To address these limitations, we propose DABUF, leveraging data attribution to detect and filter unsafe training data by attributing harmful model outputs to influential training data points. DABUF enables flexible identification of various unsafe data types without predefined taxonomies. However, in practice, model outputs can be complex with combined safe linguistic features and unsafe content, leading to reduced attribution accuracy. In such cases, DABUF will integrate moderation classifiers to identify a minimal subset of unsafe training data for targeted attribution (such as jailbreak). When model outputs are relatively straightforward, DABUF uses model outputs directly as the attribution targets. We evaluate the performance on two different tasks: in filtering jailbreaking training data and in identifying and mitigating gender bias. DABUF outperforms SOTA approaches by up to 7.5\% in detection AUPRC in jailbreaking scenarios, and 44.1\% in detecting gender bias. Moreover, retraining on DABUF-filtered data leads to higher model safety across experiments, underscoring its versatility in addressing a broad spectrum of unsafe data issues.

Related papers

• Training on the Benchmark Is Not All You Need [52.01920740114261]
  We propose a simple and effective data leakage detection method based on the contents of multiple-choice options. Our method is able to work under black-box conditions without access to model training data or weights. We evaluate the degree of data leakage of 31 mainstream open-source LLMs on four benchmark datasets.
  arXiv  Detail & Related papers  (2024-09-03T11:09:44Z)
• Unlearnable Examples Detection via Iterative Filtering [84.59070204221366]
  Deep neural networks are proven to be vulnerable to data poisoning attacks. It is quite beneficial and challenging to detect poisoned samples from a mixed dataset. We propose an Iterative Filtering approach for UEs identification.
  arXiv  Detail & Related papers  (2024-08-15T13:26:13Z)
• Safe Training with Sensitive In-domain Data: Leveraging Data Fragmentation To Mitigate Linkage Attacks [2.8186733524862158]
  Current text generation models are trained using real data which can potentially contain sensitive information. We propose a safer alternative which sees fragmented data in the form of domain-specific short phrases randomly grouped together.
  arXiv  Detail & Related papers  (2024-04-30T12:09:55Z)
• Enabling Privacy-Preserving Cyber Threat Detection with Federated Learning [4.475514208635884]
  This study systematically profiles the (in)feasibility of learning for privacy-preserving cyber threat detection in terms of effectiveness, byzantine resilience, and efficiency. It shows that FL-trained detection models can achieve a performance that is comparable to centrally trained counterparts. Under a realistic threat model, FL turns out to be adversary-resistant to attacks of both data poisoning and model poisoning.
  arXiv  Detail & Related papers  (2024-04-08T01:16:56Z)
• From Zero to Hero: Detecting Leaked Data through Synthetic Data Injection and Model Querying [10.919336198760808]
  We introduce a novel methodology to detect leaked data that are used to train classification models. textscLDSS involves injecting a small volume of synthetic data--characterized by local shifts in class distribution--into the owner's dataset. This enables the effective identification of models trained on leaked data through model querying alone.
  arXiv  Detail & Related papers  (2023-10-06T10:36:28Z)
• Enhancing Multiple Reliability Measures via Nuisance-extended Information Bottleneck [77.37409441129995]
  In practical scenarios where training data is limited, many predictive signals in the data can be rather from some biases in data acquisition. We consider an adversarial threat model under a mutual information constraint to cover a wider class of perturbations in training. We propose an autoencoder-based training to implement the objective, as well as practical encoder designs to facilitate the proposed hybrid discriminative-generative training.
  arXiv  Detail & Related papers  (2023-03-24T16:03:21Z)
• SRoUDA: Meta Self-training for Robust Unsupervised Domain Adaptation [25.939292305808934]
  Unsupervised domain adaptation (UDA) can transfer knowledge learned from rich-label dataset to unlabeled target dataset. In this paper, we present a new meta self-training pipeline, named SRoUDA, for improving adversarial robustness of UDA models.
  arXiv  Detail & Related papers  (2022-12-12T14:25:40Z)
• CAFA: Class-Aware Feature Alignment for Test-Time Adaptation [50.26963784271912]
  Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. We propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously encourages a model to learn target representations in a class-discriminative manner.
  arXiv  Detail & Related papers  (2022-06-01T03:02:07Z)
• Gradient-based Data Subversion Attack Against Binary Classifiers [9.414651358362391]
  In this work, we focus on label contamination attack in which an attacker poisons the labels of data to compromise the functionality of the system. We exploit the gradients of a differentiable convex loss function with respect to the predicted label as a warm-start and formulate different strategies to find a set of data instances to contaminate. Our experiments show that the proposed approach outperforms the baselines and is computationally efficient.
  arXiv  Detail & Related papers  (2021-05-31T09:04:32Z)
• How Robust are Randomized Smoothing based Defenses to Data Poisoning? [66.80663779176979]
  We present a previously unrecognized threat to robust machine learning models that highlights the importance of training-data quality. We propose a novel bilevel optimization-based data poisoning attack that degrades the robustness guarantees of certifiably robust classifiers. Our attack is effective even when the victim trains the models from scratch using state-of-the-art robust training methods.
  arXiv  Detail & Related papers  (2020-12-02T15:30:21Z)
• Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Training [55.824641135682725]
  Domain adaptation experiments using WSJ as a source domain and TED-LIUM 3 as well as SWITCHBOARD show that up to 80% of the performance of a system trained on ground-truth data can be recovered.
  arXiv  Detail & Related papers  (2020-11-26T18:51:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.