Ensembling LLMs for AI-Augmented Cybersecurity Software Requirements Generation

• URL: http://arxiv.org/abs/2609.10316v1
• Date: Wed, 09 Sep 2026 15:22:20 GMT
• Title: Ensembling LLMs for AI-Augmented Cybersecurity Software Requirements Generation
• Abstract summary: Large language models (LLMs) can accelerate cybersecurity requirements engineering, but any single run is unreliable.<n>We study post-generation ensembling, aggregating runs with information-retrieval data-fusion operators.<n>We propose two strategies: Uniform fusion rewards mere cross-run agreement, whereas Naive-Bayes fusion weights each run by its estimated reliability.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Translating high-level controls from security standards into concrete, system-specific requirements is central to cybersecurity requirements engineering. Large language models (LLMs) can accelerate this labor-intensive, recall-sensitive task, but any single run is unreliable: it misses valid safeguards while introducing plausible hallucinations, and outputs shift across runs and models. We reframe this variability as a resource: rather than selecting one output, we study post-generation ensembling, aggregating stochastic runs with information-retrieval data-fusion operators. We propose two strategies: Uniform fusion rewards mere cross-run agreement, whereas Naive-Bayes fusion weights each run by its estimated reliability. We evaluate both over 24 runs from 12 configurations across four model families, generated for ten ISO/IEC 27002:2022 controls and expert-judged against a gold standard of 72 valid requirements. Pooling every run's output recovers all 72 (whereas single configurations recover on average under half) but also 111 hallucinations. Fusion separates the wheat from the chaff, ranking valid requirements well ahead of hallucinations. In the areas under the precision-recall and ROC curves, Uniform fusion alone largely surpasses every original run and configuration by 0.142 and 0.118 over the best configuration. Naive-Bayes weighting adds a further 0.039 and 0.052, reaching 0.864 and 0.869 while attaining useful operating points earlier. Internal validation confirms the stability of these gains: they stay positive in at least 92% of out-of-bag bootstrap resamples and every structured-perturbation sample. Post-generation fusion thus turns apparent noise into a practical asset: a lightweight layer giving analysts broader coverage and a better prioritized review queue, using affordable, below-frontier models alone.

Related papers

• SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution [6.908637308550535]
  SEVA is a structured verification agent that emits evidence alignments, step-by-step reasoning chains, calibrated confidence, and a six-category error diagnosis.<n>On ClearFacts, SEVA-3B matches GPT-4o-mini (69.0 vs. 69.8 F1) while producing substantially richer, auditable output.
  arXiv  Detail & Related papers  (2026-06-29T02:37:13Z)
• The Capability Frontier: Benchmarks Miss 82% of Model Performance [7.258179693117413]
  Existing benchmarks typically report accuracy for a single model on a single run.<n>This systematically understates real-world LLM capabilities, particularly under heterogeneous data distributions.
  arXiv  Detail & Related papers  (2026-06-25T10:20:47Z)
• Evidence-Grounded Ensemble Diagnosis of 802.11 Packet Captures: A Multi-Stage Pipeline with Deterministic Reliability Scoring [1.0170129555792935]
  Diagnosing 802.11 packet captures requires expert protocol knowledge, is slow, inconsistent across engineers, and unscalable.<n>LLM-based approaches sound plausible but fabricate protocol events absent from captures, produce uncalibrated confidence scores, and suffer evaluation bias when golden references are co-produced by the model under test.<n>We introduce PROBE, a multi-stage pipeline addressing all three failures.
  arXiv  Detail & Related papers  (2026-06-05T03:39:58Z)
• TAME: Test-Time Adversarial Prompt Tuning via Mixture-of-Experts for Vision-Language Models [76.92890872338235]
  Large-scale pre-trained Vision-Language models (VLMs) exhibit strong zero-shot generalization, yet are highly vulnerable to imperceptible adversarial perturbations.<n>To enhance robustness without requiring downstream task-specific retraining, we propose TAME, a novel test-time defense.
  arXiv  Detail & Related papers  (2026-05-17T18:07:08Z)
• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents [66.97968363332465]
  We introduce Claw-Eval, an end-to-end evaluation suite addressing all three gaps in agent benchmarks.<n>Claw-Eval comprises 300 human-verified tasks spanning 9 categories across three groups.<n>Every agent action is recorded through three independent evidence channels.
  arXiv  Detail & Related papers  (2026-04-07T17:43:18Z)
• BadCLIP++: Stealthy and Persistent Backdoors in Multimodal Contrastive Learning [73.46118996284888]
  Research on backdoor attacks against multimodal contrastive learning models faces two key challenges: stealthiness and persistence.<n>We propose BadCLIP++, a unified framework that tackles both challenges.<n>For stealthiness, we introduce a semantic-fusion QR micro-trigger that embeds imperceptible patterns near task-relevant regions.<n>For persistence, we stabilize trigger embeddings via radius shrinkage and centroid alignment.
  arXiv  Detail & Related papers  (2026-02-19T08:31:16Z)
• CVeDRL: An Efficient Code Verifier via Difficulty-aware Reinforcement Learning [57.24524263804788]
  Code verifiers play a critical role in post-verification for LLM-based code generation.<n>Existing supervised fine-tuning methods suffer from data scarcity, high failure rates, and poor inference efficiency.<n>We show that naive RL with only functionality rewards fails to generate effective unit tests for difficult branches and samples.
  arXiv  Detail & Related papers  (2026-01-30T10:33:29Z)
• When Fine-Tuning is Not Enough: Lessons from HSAD on Hybrid and Adversarial Audio Spoof Detection [3.7411108810335922]
  Spoof detection is a challenge for voice authentication, smart assistants, and telecom security.<n>We present a benchmark containing 1,248 clean and 41,044 degraded utterances across four classes: human, cloned, zero-shot AI-generated, and hybrid audio.<n>Results reveal critical lessons: pretrained models overgeneralize and collapse under hybrid conditions; spoof-specific fine-tuning improves separability but struggles with unseen compositions.
  arXiv  Detail & Related papers  (2025-09-09T01:43:28Z)
• VAULT: Vigilant Adversarial Updates via LLM-Driven Retrieval-Augmented Generation for NLI [15.320553375828045]
  VAULT is a fully automated adversarial RAG pipeline that uncovers and remedies weaknesses in NLI models.<n>VAULT consistently outperforms prior in-context adversarial methods by up to 2.0% across datasets.
  arXiv  Detail & Related papers  (2025-08-01T14:22:54Z)
• Precise Information Control in Long-Form Text Generation [107.83171105517863]
  We propose Precise Information Control (PIC), a new task formulation that requires models to generate long-form outputs grounded in a provided set of short self-contained statements.<n> PIC includes a full setting that tests a model's ability to include exactly all input claims, and a partial setting that requires the model to selectively incorporate only relevant claims.<n>We present PIC-Bench, a benchmark of eight long-form generation tasks adapted to the PIC setting, where LMs are supplied with well-formed, verifiable input claims.
  arXiv  Detail & Related papers  (2025-06-06T23:42:42Z)
• Cream of the Crop: Harvesting Rich, Scalable and Transferable Multi-Modal Data for Instruction Fine-Tuning [59.56171041796373]
  We harvest multi-modal instructional data in a robust and efficient manner.<n>We take interaction style as diversity indicator and use a multi-modal rich styler to identify data instruction patterns.<n>Across 10+ experimental settings, validated by 14 multi-modal benchmarks, we demonstrate consistent improvements over random sampling, baseline strategies and state-of-the-art selection methods.
  arXiv  Detail & Related papers  (2025-03-17T17:11:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.