Epidemiology of Model Collapse: Modeling Synthetic Data Contamination via Bilayer SIR Dynamics

• URL: http://arxiv.org/abs/2606.05168v1
• Date: Tue, 14 Apr 2026 06:49:30 GMT
• Title: Epidemiology of Model Collapse: Modeling Synthetic Data Contamination via Bilayer SIR Dynamics
• Abstract summary: Training on synthetic data causes model collapse, but existing analyses treat this as single-chain degradation.<n>We propose a bilayer coupled SIR/SIRS framework treating data corpora and AI models as two interacting populations.
• Score: 1.7832408661523071
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Training on synthetic data causes model collapse, but existing analyses treat this as single-chain degradation. In reality, the AI ecosystem involves cross-contamination: models ingest synthetic data from other models, produce new synthetic text, and contaminate shared corpora. We propose a bilayer coupled SIR/SIRS framework -- a phenomenological mean-field model treating data corpora and AI models as two interacting populations, each with susceptible, infected, and recovered compartments linked by cross-layer transmission. The SIRS variant (our primary recommendation) incorporates immunity waning, reflecting that filtered corpora and retrained models remain susceptible to re-contamination. We derive the basic reproduction number $R_0 = \sqrt{β_D β_M / [(γ_D+μ_D)(γ_M+μ_M)]}$ via the Next Generation Matrix and apply standard epidemic threshold results to the bilayer system. Illustrative scenario-based calibration from public AI text prevalence data yields supercritical dynamics ($R_0 > 1$) across three scenarios; Sobol sensitivity analysis identifies synthetic-text detection as the highest-leverage parameter. A bipartite-network agent-based model confirms mean-field consistency ($R^2 > 0.96$) for dense networks but degrades under heterogeneity. GPT-2 contamination chain experiments (192 runs across WikiText and Shakespeare) show dose-response degradation and diversity loss qualitatively consistent with the threshold picture. Matched-budget source-diversity experiments (1,088 runs) provide suggestive evidence that multi-source mixing modestly attenuates collapse, but the effect vanishes at lower contamination fractions. Intervention analysis identifies detection-based filtering and herd immunity as the highest-leverage strategies.

Related papers

• Few-Shot Synthetic Data Generation with Diffusion Models for Downstream Vision Tasks [36.94429692322632]
  Class imbalance is a persistent challenge in visual recognition.<n>We propose a lightweight synthetic data augmentation pipeline that fine-tunes a LoRA adapter on as few as 20-50 real images of a rare class.
  arXiv  Detail & Related papers  (2026-05-12T10:11:57Z)
• SynSur: An end-to-end generative pipeline for synthetic industrial surface defect generation and detection [4.219862887901752]
  We present an end-to-end pipeline for synthetic defect generation and annotation.<n>We analyze key stages of the pipeline, including prompt construction, LoRA selection, and sample filtering.<n>Overall, the paper provides an end-to-end assessment of diffusion-based industrial defect synthesis.
  arXiv  Detail & Related papers  (2026-04-29T12:57:32Z)
• SNPgen: Phenotype-Supervised Genotype Representation and Synthetic Data Generation via Latent Diffusion [6.660968772645983]
  SNPgen is a conditional latent diffusion framework for generating phenotype-supervised synthetic genotypes.<n> evaluated on UK Biobank individuals across four complex diseases.
  arXiv  Detail & Related papers  (2026-03-11T15:23:37Z)
• Towards Syn-to-Real IQA: A Novel Perspective on Reshaping Synthetic Data Distributions [74.00222571094437]
  Blind Image Quality Assessment (BIQA) has advanced significantly through deep learning, but the scarcity of large-scale labeled datasets remains a challenge.<n>We make a key observation that representations learned from synthetic datasets often exhibit a discrete and clustered pattern that hinders regression performance.<n>We introduce a novel framework SynDR-IQA, which reshapes synthetic data distribution to enhance BIQA generalization.
  arXiv  Detail & Related papers  (2026-01-01T06:11:16Z)
• BURN: Backdoor Unlearning via Adversarial Boundary Analysis [73.14147934175604]
  Backdoor unlearning aims to remove backdoor-related information while preserving the model's original functionality.<n>We propose Backdoor Unlearning via adversaRial bouNdary analysis (BURN), a novel defense framework that integrates false correlation decoupling, progressive data refinement, and model purification.
  arXiv  Detail & Related papers  (2025-07-14T17:13:06Z)
• Unlasting: Unpaired Single-Cell Multi-Perturbation Estimation by Dual Conditional Diffusion Implicit Bridges [68.98973318553983]
  We propose a framework based on Dual Diffusion Implicit Bridges (DDIB) to learn the mapping between different data distributions.<n>We integrate gene regulatory network (GRN) information to propagate perturbation signals in a biologically meaningful way.<n>We also incorporate a masking mechanism to predict silent genes, improving the quality of generated profiles.
  arXiv  Detail & Related papers  (2025-06-26T09:05:38Z)
• Leveraging Diffusion Models for Synthetic Data Augmentation in Protein Subcellular Localization Classification [0.0]
  We implement a class-conditional denoising diffusion probabilistic model (DDPM) to produce label-consistent samples.<n>We explore their integration with real data via two hybrid training strategies: Mix Loss and Mix Representation.<n>Our findings highlight the importance of realistic data generation and robust supervision when incorporating generative augmentation into biomedical image classification.
  arXiv  Detail & Related papers  (2025-05-28T22:58:50Z)
• Contrastive Model Inversion for Data-Free Knowledge Distillation [60.08025054715192]
  We propose Contrastive Model Inversion, where the data diversity is explicitly modeled as an optimizable objective. Our main observation is that, under the constraint of the same amount of data, higher data diversity usually indicates stronger instance discrimination. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that CMI achieves significantly superior performance when the generated data are used for knowledge distillation.
  arXiv  Detail & Related papers  (2021-05-18T15:13:00Z)
• Robust Finite Mixture Regression for Heterogeneous Targets [70.19798470463378]
  We propose an FMR model that finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The results show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-10-12T03:27:07Z)
• Understanding Self-supervised Learning with Dual Deep Networks [74.92916579635336]
  We propose a novel framework to understand contrastive self-supervised learning (SSL) methods that employ dual pairs of deep ReLU networks. We prove that in each SGD update of SimCLR with various loss functions, the weights at each layer are updated by a emphcovariance operator. To further study what role the covariance operator plays and which features are learned in such a process, we model data generation and augmentation processes through a emphhierarchical latent tree model (HLTM)
  arXiv  Detail & Related papers  (2020-10-01T17:51:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.