Mapping CVEs to MITRE ATT&CK Techniques: A Curated Gold-Set Classifier and the Limits of LLM-Assisted Label Expansion

• URL: http://arxiv.org/abs/2607.25572v1
• Date: Tue, 28 Jul 2026 10:59:04 GMT
• Title: Mapping CVEs to MITRE ATT&CK Techniques: A Curated Gold-Set Classifier and the Limits of LLM-Assisted Label Expansion
• Abstract summary: We train a multi-label classifier on a curated gold dataset of 1,207 CVEs from expert MITRE Center for Threat-Informed Defense mappings.<n>The resulting model approximately doubles recall@5 compared with a zero-shot embedding-similarity baseline.<n>We then investigate whether LLM-assisted labeling can extend the gold dataset.
• Score: 46.262619407930266
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We present a reproducible pipeline for mapping Common Vulnerabilities and Exposures (CVEs) to MITRE ATT&CK Enterprise techniques from free-text vulnerability descriptions. Rather than relying on the CWE->CAPEC->ATT&CK derivation chain, whose table-expansion artifacts we quantify, we train a multi-label classifier on a curated gold dataset of 1,207 CVEs from expert MITRE Center for Threat-Informed Defense mappings. The resulting model approximately doubles recall@5 compared with a zero-shot embedding-similarity baseline and improves every ranking metric. We then investigate whether LLM-assisted labeling can extend the gold dataset. Initial experiments suggest contradictory conclusions: a single run indicates degraded performance, while averaging over five random seeds suggests a small gain. However, an independent replication and an expansion-size study (100--984 additional CVEs) show that the apparent improvement is an evaluation artifact. LLM-generated labels, with approximately 0.39 agreement with expert annotations, provide no reliable improvement at any expansion size and reduce rare-technique coverage at around 1,000 added CVEs (macro-F1 decreases by 0.04). The root cause is evaluation noise. Selecting checkpoints on a small test split effectively maximizes over many noisy evaluations, producing recall@5 differences of up to 0.05 between otherwise identical runs. Using a corrected protocol based on validation-split checkpoint selection, the gold-only model achieves recall@5 of (0.673 \pm 0.019), and repeating the decisive experiment confirms the null result for LLM expansion. A final scaling study shows that additional expert-curated data consistently improves performance, whereas LLM-labeled data does not, indicating that the classifier is limited by label quality rather than dataset size. All datasets, models, code, and training logs are publicly released.

Related papers

• Provenance, Not Behaviour: A Serialisation Artifact in Edge-IIoTset and a Leakage-Free Benchmark for Precision-Agriculture Intrusion Detection [0.0]
  Edge-IIoTset is the benchmark for machine-learning intrusion detection in the Internet of Things.<n>We show that much of that performance is not intrusion detection.<n>We produce AgriEdge: 1,276,122 rows, five devices with full attribution, and no column separating the classes above 0.0288.
  arXiv  Detail & Related papers  (2026-08-16T14:30:45Z)
• Quality-Diversity Stress Tests for Process Reward Models:What Archive Coverage Can and Cannot Certify [1.0742675209112622]
  We formulate PRM stress testing as a quality-diversity search problem using MAP-Elites.<n>We characterize what such archives certify: finite-cell repair bounds covered-cell tail risk and average residual severity but cannot bound the worst remaining cell from covered fraction alone.<n>A predeclared paired LoRA repair protocol reduces exploit rates from 0.148 to 0.037 to 0.074, the worst attack from 0.333 to 0.177 to 0.212, improves ranking AUROC without degrading best-of-4 accuracy.
  arXiv  Detail & Related papers  (2026-08-08T08:41:06Z)
• Beyond Shapley: An Influence-Based Data Auditing Pipeline for LLM Alignment and Evaluation [2.733105115068754]
  We introduce a scalable, inference-only data valuation pipeline that approximates the Shapley value without iterative model retraining.<n>We demonstrate the pipeline's efficacy in sanitizing two heavily vetted alignment datasets.
  arXiv  Detail & Related papers  (2026-07-24T03:31:44Z)
• CLAP: Closed-Loop Training, Evaluation, and Release Control for Domain Agent Post-training [14.51193578486111]
  CLAP converts business data into structured SFT samples, decision-preference samples, holdout sets, risk diagnostics, and release-gate records.<n>On five anonymized manufacturing-scenario batches, QLoRA-style LoRA-SFT yields modest average gains.
  arXiv  Detail & Related papers  (2026-07-02T08:07:56Z)
• Metric-Dependent Annotation Saturation for Learning from Label Distributions [0.0]
  We fine-tune NLI models on label distributions subsampled from ChaosNLI.<n>In our 3-class NLI setting, entropy correlation requires N 20-50 annotators to converge.<n>Soft labels carry item-specific signal that label smoothing cannot replicate.
  arXiv  Detail & Related papers  (2026-05-28T11:46:51Z)
• Beyond the Fold: Quantifying Split-Level Noise and the Case for Leave-One-Dataset-Out AU Evaluation [1.7159330469510958]
  We show that cross-validation itself introduces measurable variance.<n>On BP4D+, repeated 3-fold subject-exclusive splits produce an empirical noise floor of $pm 0.065$ in average F1.<n>We further evaluate cross-dataset robustness using a Leave-One-Dataset-Out protocol across five AU datasets.
  arXiv  Detail & Related papers  (2026-04-02T15:28:55Z)
• Deterministic Fuzzy Triage for Legal Compliance Classification and Evidence Retrieval [0.0]
  Legal teams increasingly use machine learning to triage large volumes of contractual evidence.<n>Many models are opaque, non-deterministic, and difficult to align with frameworks such as HIPAA or NERC-CIP.<n>We study a simple, reproducible alternative based on deterministic dual encoders and transparent fuzzy triage bands.
  arXiv  Detail & Related papers  (2026-03-08T00:31:34Z)
• MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark [57.999567012489706]
  We propose a contamination-free and more challenging benchmark called MMLU-CF.<n>This benchmark reassesses LLMs' understanding of world knowledge by averting both unintentional and malicious data leakage.<n>Our evaluation of mainstream LLMs reveals that the powerful GPT-4o achieves merely a 5-shot score of 73.4% and a 0-shot score of 71.9% on the test set.
  arXiv  Detail & Related papers  (2024-12-19T18:58:04Z)
• LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond [135.8013388183257]
  We propose a new protocol for inconsistency detection benchmark creation and implement it in a 10-domain benchmark called SummEdits. Most LLMs struggle on SummEdits, with performance close to random chance. The best-performing model, GPT-4, is still 8% below estimated human performance.
  arXiv  Detail & Related papers  (2023-05-23T21:50:06Z)
• Neighborhood Collective Estimation for Noisy Label Identification and Correction [92.20697827784426]
  Learning with noisy labels (LNL) aims at designing strategies to improve model performance and generalization by mitigating the effects of model overfitting to noisy labels. Recent advances employ the predicted label distributions of individual samples to perform noise verification and noisy label correction, easily giving rise to confirmation bias. We propose Neighborhood Collective Estimation, in which the predictive reliability of a candidate sample is re-estimated by contrasting it against its feature-space nearest neighbors.
  arXiv  Detail & Related papers  (2022-08-05T14:47:22Z)
• OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers [71.08167292329028]
  We propose a novel Open-set Semi-Supervised Learning (OSSL) approach called OpenMatch. OpenMatch unifies FixMatch with novelty detection based on one-vs-all (OVA) classifiers. It achieves state-of-the-art performance on three datasets, and even outperforms a fully supervised model in detecting outliers unseen in unlabeled data on CIFAR10.
  arXiv  Detail & Related papers  (2021-05-28T23:57:15Z)
• In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning [53.1047775185362]
  Pseudo-labeling (PL) is a general SSL approach that does not have this constraint but performs relatively poorly in its original formulation. We argue that PL underperforms due to the erroneous high confidence predictions from poorly calibrated models. We propose an uncertainty-aware pseudo-label selection (UPS) framework which improves pseudo labeling accuracy by drastically reducing the amount of noise encountered in the training process.
  arXiv  Detail & Related papers  (2021-01-15T23:29:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.