VESTIGE: A Knowledge-Guided Masking Strategy for Corruption-Aware Fine-Tuning of Genomic Transformers, Validated on Ancient DNA Reconstruction

• URL: http://arxiv.org/abs/2607.27712v1
• Date: Thu, 30 Jul 2026 05:47:47 GMT
• Title: VESTIGE: A Knowledge-Guided Masking Strategy for Corruption-Aware Fine-Tuning of Genomic Transformers, Validated on Ancient DNA Reconstruction
• Abstract summary: Masked--model fine-tuning applies a uniform masking probability across every token position, assuming reconstruction difficulty is position-agnostic.<n>We introduce VESTIGE, a parameter-free, drop-in replacement for the standard collator that aligns the masking distribution with an empirically measured per-position corruption profile.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Standard masked-language-model fine-tuning applies a uniform masking probability across every token position, assuming reconstruction difficulty is position-agnostic. When the degradation process is characterised and concentrated at predictable positions, this assumption fails: at peak damage sites the model can underperform a frequency-matched random predictor. We introduce VESTIGE, a parameter-free, drop-in replacement for the standard MLM collator that aligns the masking distribution with an empirically measured per-position corruption profile. We apply it to ancient DNA (aDNA) reconstruction, where cytosine deamination produces a position-dependent C-to-T / G-to-A gradient quantified per-position by mapDamage2. Rescaling so the mean C/G masking rate equals 15% - identical to standard MLM - isolates spatial redistribution as the sole variable, with model, data, seed, and hyperparameters held fixed across both DNABERT-2 runs on a mammoth CDS corpus (two specimens, seven genes). Across six terminal-zone widths and 626 paired windows, VESTIGE leads standard MLM at every width (Delta = +4.18 to +10.35 pp, all p < 10^-8), cuts validation cross-entropy by 13% (3.274 vs. 3.757), and yields ESMFold reconstructions with TM-score > 0.95 across all six reconstructions (three genes) even under damage amplified 10-30x beyond authentic PMD rates. A 1D CNN biosecurity classifier returns AUC = 0.935 and clears 98.2% of reconstructed windows, the 1.76% remainder attributable to reference-genome features, not reconstruction artefacts. The principle is domain-agnostic: any measurable position- or context-specific corruption profile - FFPE, bisulfite, metagenomic, or nanopore - substitutes directly for the PMD array, making VESTIGE a knowledge-guided training routine for intelligent systems operating on degraded or noisy sequence inputs.

Related papers

• SeGDeP: Semantic- and Geometric-Aware Decoupled Prompts for Reasoning Segmentation [51.46828526392219]
  Reasoning segmentation converts an implicit linguistic conclusion into a precise mask.<n>Existing MLLM-segmenter interfaces either use a special trigger or compress both signals into one context.<n>We present SeGDeP, an explicit what-where interface.
  arXiv  Detail & Related papers  (2026-09-08T15:11:32Z)
• When Genomic Masking Priors Fail to Transfer: Strong Variant Prediction, Weak Functional Generation [6.888870145233493]
  Bidirectional discrete diffusion model appears naturally suited to genomic modeling because it can reconstruct missing sequence from both flanks.<n>We developed GenDA (Genomic Density-optimized Absorbing Diffusion) under the additional hypothesis that entropy-guided span placement would concentrate reconstruction pressure on compositionally complex regions.<n>Our results only partially support this premise.
  arXiv  Detail & Related papers  (2026-09-04T08:21:34Z)
• Distribution-free false-alarm calibration and chance-corrected spatial evaluation for industrial anomaly detection [2.5273986395975476]
  We combine a distribution-free upper tolerance threshold with a paired-minus-crossed spatial test.<n>We evaluate three detectors on 120 point-defect images from three ISP-AD modalities and three fixed data splits.
  arXiv  Detail & Related papers  (2026-08-15T07:21:05Z)
• CASCADE: An Agentic Regulatory Network Framework for Patient-Data-Validated Downstream Perturbation Prediction [0.0]
  We use focal-gene copy-number amplification as a proxy for the inverse of knockdown against real TCGA patient tumor data.<n>For MYC, CASCADE's predicted knockdown targets show strong concordance with real amplified-vs-non-amplified tumor expression.<n>Its accuracy is not shown to exceed existing public knowledge of MYC- or E2F-driven biology.
  arXiv  Detail & Related papers  (2026-08-05T19:30:44Z)
• Theia: Large-Scale Multimodal Captioning and Automated Validation of the Incidents1M Dataset for Data-Free Distillation [43.1212452324751]
  Existing datasets in this domain either entirely lack descriptive text, or suffer from severe text-image semantic misalignment.<n>We present a novel methodology to construct and automatically validate a large-scale multimodal dataset for disaster response.
  arXiv  Detail & Related papers  (2026-07-30T14:23:01Z)
• Post-Operative Glioma Segmentation via Loss Stabilization, Normalization and Subspace Attention [46.03321798937855]
  We show that the standard Generalized Dice Loss (GDL) is unstable under domain shift.<n>We propose a Subspace-Aware Class Attention (SACA) module that re-calibrates the bottleneck features and raises Enhancing Tumor (ET) sensitivity by 8%.
  arXiv  Detail & Related papers  (2026-07-23T09:59:30Z)
• DNA: Dual-stage Native Attribution for Generated Image Source Tracing [75.65470221927096]
  Existing proactive methods rely on watermark embedding or model modification.<n>DNA is a coarse-to-fine framework that follows this hierarchy without additional neural-network training.<n>Experiments show that DNA achieves 89.11% end-to-end attribution accuracy.
  arXiv  Detail & Related papers  (2026-07-15T10:27:52Z)
• Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems [47.027290803102666]
  Normal behaviour is the union of many imbalanced, curved, thin-fringed operating regimes.<n>We model the normal law with a jointly learned latent representation plus explicit Gaussian-mixture mode clustering.<n>We evaluate under a deliberately fair protocol: raw point-wise metrics with no point adjustment, a trivial-detector difficulty split, prevalence-matched F1, and train-normal-only calibration.
  arXiv  Detail & Related papers  (2026-07-07T10:10:00Z)
• Layer-Isolated Evaluation: Gating the Deterministic Scaffold of a Production LLM Agent with a No-LLM, Regression-Locked Test Harness [45.148328075418156]
  A deployed ordering agent is decomposed into a fixed taxonomy of layers.<n>The pure suite runs in CI on every change against a locked per-slice baseline.<n>We validate by controlled regression injection, degrading one layer at a time across seven non-safety layers.
  arXiv  Detail & Related papers  (2026-06-10T05:55:13Z)
• The Dark Regulome: Disentangling Predictability from Regulation in Genomic Foundation Models [0.0]
  High-grade gliomas integrate into neural circuits through functional synapses with neurons.<n>The regulatory program written across the dark genome is the natural substrate to probe.<n>We introduce a residualization-and-permutation diagnostic that separates predictability-driven from regulation-driven RIS variance.
  arXiv  Detail & Related papers  (2026-06-05T02:20:12Z)
• Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation [6.129872931808218]
  Key-value ( KV) cache quantization is widely used to reduce Large Language Model (LLM) inference memory.<n>In this study, we explore alignment preservation under KV cache quantization.<n>We find that low-bit quantization can silently destroy safety alignment.
  arXiv  Detail & Related papers  (2026-06-01T02:02:20Z)
• Fragile Reasoning: A Mechanistic Analysis of LLM Sensitivity to Meaning-Preserving Perturbations [2.7946918847372277]
  Large language models demonstrate strong performance on mathematical reasoning benchmarks, yet remain surprisingly fragile to meaning-preserving surface perturbations.<n>We systematically evaluate three open-weight LLMs, Mistral-7B, Llama-3-8B, and Qwen2.5-7B, on 677 GSM8K problems paired with semantically equivalent variants.<n>All three models exhibit substantial answer-flip rates (28.8%-45.1%), with number paraphrasing consistently more disruptive than name swaps.
  arXiv  Detail & Related papers  (2026-04-02T05:30:20Z)
• DAONet-YOLOv8: An Occlusion-Aware Dual-Attention Network for Tea Leaf Pest and Disease Detection [2.661320179262946]
  We propose an enhanced YOLOv8 variant with three key improvements to accurately detect tea leaf pests and diseases.<n>ExistingNet-YOLOv8 achieves 92.97% precision, 92.80% recall, 97.10% mAP@50 and 76.90% mAP@50:95, outperforming the YOLOv8n baseline by 2.34, 4.68, 1.40 and 1.80 percentage points respectively.
  arXiv  Detail & Related papers  (2025-11-28T14:28:30Z)
• DNABERT-2: Fine-Tuning a Genomic Language Model for Colorectal Gene Enhancer Classification [0.0]
  DNABERT-2 is a transformer genomic language model that uses byte-pair encoding to learn variable-length tokens from DNA.<n>Gene enhancers control when and where genes switch on, yet their sequence diversity and tissue specificity make them hard to pinpoint in colorectal cancer.<n>This is the first study to apply a second-generation genomic language model with BPE tokenization to enhancer classification in colorectal cancer.
  arXiv  Detail & Related papers  (2025-09-28T16:10:03Z)
• Backdoor Cleaning without External Guidance in MLLM Fine-tuning [76.82121084745785]
  Believe Your Eyes (BYE) is a data filtering framework that leverages attention entropy patterns as self-supervised signals to identify and filter backdoor samples.<n>It achieves near-zero attack success rates while maintaining clean-task performance.
  arXiv  Detail & Related papers  (2025-05-22T17:11:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.