Pretraining Data Can Be Poisoned through Computational Propaganda

• URL: http://arxiv.org/abs/2607.15267v1
• Date: Thu, 16 Jul 2026 17:56:05 GMT
• Title: Pretraining Data Can Be Poisoned through Computational Propaganda
• Abstract summary: Poisoning pretraining data can introduce harmful behaviors to LMs difficult to detect and mitigate.<n>We demonstrate that poisoning attacks on pretraining data are feasible through an existing web-scale content injection mechanism.<n>We use HalfLife to explore the feasibility of poisoning pretraining corpora at web scale through open discussion interfaces.
• Score: 94.12810824760635
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Poisoning pretraining data can introduce harmful behaviors to LMs that are difficult to detect and mitigate. Prior work on poisoning pretraining data has largely exploited established data sources such as Wikipedia, which do not represent the large scale and heterogeneity typical of pretraining corpora, and has ignored the interaction between poisoned data and data curation pipelines. We demonstrate that poisoning attacks on pretraining data are feasible beyond this limited setting through an existing web-scale content injection mechanism: public discussion interfaces. Additionally, to measure whether malicious content is included after web crawling and data curation, we introduce HalfLife, a novel analysis for estimating adversarial content inclusion in web-crawl based LM training data. We use HalfLife to explore the feasibility of poisoning pretraining corpora at web scale through open discussion interfaces. Our analysis demonstrates the importance of estimating whether poison injections are included in pretraining data, and establishes third-party webpage content as a possible vector for attacking language model pretraining.

Related papers

• Unlearned but Not Forgotten: Data Extraction after Exact Unlearning in LLM [31.093224824043087]
  We introduce a novel data extraction attack that leverages signals from the pre-unlearning model to guide the post-unlearning model.<n>We demonstrate our attack's effectiveness on a simulated medical diagnosis dataset to highlight real-world privacy risks associated with exact unlearning.
  arXiv  Detail & Related papers  (2025-05-30T09:09:33Z)
• Understanding and Mitigating Toxicity in Image-Text Pretraining Datasets: A Case Study on LLaVA [0.0]
  This dataset removes 7,531 of toxic image-text pairs in the LLaVA pre-training dataset.<n>We offer guidelines for implementing robust toxicity detection pipelines.
  arXiv  Detail & Related papers  (2025-05-09T18:01:50Z)
• Towards Safer Pretraining: Analyzing and Filtering Harmful Content in Webscale datasets for Responsible LLMs [1.7451266777840306]
  Large language models (LLMs) have become integral to various real-world applications, leveraging massive, web-sourced datasets like Common Crawl, C4, and FineWeb for pretraining.<n>Training LLMs on such unfiltered data risks perpetuating toxic behaviors, spreading misinformation, and amplifying societal biases.<n>This paper presents a large-scale analysis of inappropriate content across these datasets, offering a comprehensive taxonomy that categorizes harmful webpages into Topical and Toxic based on their intent.
  arXiv  Detail & Related papers  (2025-05-04T06:37:20Z)
• Persistent Pre-Training Poisoning of LLMs [71.53046642099142]
  Our work evaluates for the first time whether language models can also be compromised during pre-training. We pre-train a series of LLMs from scratch to measure the impact of a potential poisoning adversary. Our main result is that poisoning only 0.1% of a model's pre-training dataset is sufficient for three out of four attacks to persist through post-training.
  arXiv  Detail & Related papers  (2024-10-17T16:27:13Z)
• Data Poisoning and Leakage Analysis in Federated Learning [10.090442512374661]
  Data poisoning and leakage risks impede the massive deployment of federated learning in the real world. This chapter reveals the truths and pitfalls of understanding two dominating threats: em training data privacy intrusion and em training data poisoning
  arXiv  Detail & Related papers  (2024-09-19T16:50:29Z)
• On the Exploitability of Instruction Tuning [103.8077787502381]
  In this work, we investigate how an adversary can exploit instruction tuning to change a model's behavior. We propose textitAutoPoison, an automated data poisoning pipeline. Our results show that AutoPoison allows an adversary to change a model's behavior by poisoning only a small fraction of data.
  arXiv  Detail & Related papers  (2023-06-28T17:54:04Z)
• Autoregressive Perturbations for Data Poisoning [54.205200221427994]
  Data scraping from social media has led to growing concerns regarding unauthorized use of data. Data poisoning attacks have been proposed as a bulwark against scraping. We introduce autoregressive (AR) poisoning, a method that can generate poisoned data without access to the broader dataset.
  arXiv  Detail & Related papers  (2022-06-08T06:24:51Z)
• Accumulative Poisoning Attacks on Real-time Data [56.96241557830253]
  We show that a well-designed but straightforward attacking strategy can dramatically amplify the poisoning effects. Our work validates that a well-designed but straightforward attacking strategy can dramatically amplify the poisoning effects.
  arXiv  Detail & Related papers  (2021-06-18T08:29:53Z)
• Weight Poisoning Attacks on Pre-trained Models [103.19413805873585]
  We show that it is possible to construct weight poisoning'' attacks where pre-trained weights are injected with vulnerabilities that expose backdoors'' after fine-tuning. Our experiments on sentiment classification, toxicity detection, and spam detection show that this attack is widely applicable and poses a serious threat.
  arXiv  Detail & Related papers  (2020-04-14T16:51:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.