Related papers: Cybersecurity Detection Classification with Reasoning-enabled Language Models

Cybersecurity Detection Classification with Reasoning-enabled Language Models

URL: http://arxiv.org/abs/2607.28460v1
Date: Thu, 30 Jul 2026 16:22:13 GMT
Title: Cybersecurity Detection Classification with Reasoning-enabled Language Models
Abstract summary: A major issue in Security Operations Centers (SOCs) is alert fatigue.<n>Prior work prompts or fine-tunes large language models (LLMs) to emit a triage label directly, but does not train them to reason about whether a detection is a genuine threat.<n>We train a chain-of-thought (CoT) reasoning-enabled triage classifier on real, human-labeled Windows endpoint detections.
Score: 50.72675435043546
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: A major issue in Security Operations Centers (SOCs) is alert fatigue, as the number of detections reported is more than staff can triage in a given day. Prior work prompts or fine-tunes large language models (LLMs) to emit a triage label directly, but does not train them to reason about whether a detection is a genuine threat. We train a chain-of-thought (CoT) reasoning-enabled triage classifier on real, human-labeled Windows endpoint detections by combining automated prompt optimization, self-training, and reinforcement learning with verifiable rewards. We find that CoT reasoning also degrades the label-token probabilities that automated triage relies on, so we separately train a calibrator that reads the full reasoning trace and estimates the probability that the verdict is correct. Our system reaches 82.6% test accuracy and, at the high-confidence operating point that governs automated triage, improves benign recall by 43.0% and malicious recall by 18.3% over a direct-label LLM classifier. We further show that the trained calibrator is necessary - an untrained confidence judge collapses high-confidence recall to zero - and that a finetuned 30B model significantly outperforms frontier general-purpose models, motivating targeted training over scale.

Related papers

SingGuard-NSFA: Extensible Guardrails for Agentic AI via Generative Reasoning and Real-Time Classification [0.0]
nsfaguard is a guardrail framework for securing agentic AI systems against operational threats.<n>We first introduce the NSFA taxonomy, which organizes 185 risk variants into a CIA-triad-grounded hierarchy.<n>We develop a dual-mode approach combining SFT-based generative reasoning for interpretable offline auditing with discriminative classification heads on the frozen backbone.
arXiv Detail & Related papers (2026-07-13T07:28:45Z)
Teaching LLMs Program Semantics via Symbolic Execution Traces [0.7046782561282057]
We introduce an evaluation framework of 500 C verification tasks built on SV-COMP 2025.<n>We evaluate 14 models across six families and find high overall accuracy masks a critical weakness.<n>Just $sim$3,000 bug traces combined with chain-of-thought reasoning at inference time improve violation detection by over 17 percentage points.
arXiv Detail & Related papers (2026-05-07T13:01:06Z)
Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models [94.68358825189738]
We propose a post-training framework for verifiable domains that jointly optimize prediction accuracy and reasoning quality.<n>We evaluate VPS on chess, a controlled testbed where reasoning steps can be deterministically verified against engine signals.<n>VPS preserves accuracy while significantly improving reasoning quality, reducing win-rate error by up to 30% and restoring consistency to near saturation.
arXiv Detail & Related papers (2026-04-03T15:19:46Z)
CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution [52.691495954442985]
CoVerRL is a framework where a single model alternates between generator and verifier roles, with each capability bootstrapping the other.<n> Experiments across Qwen and Llama model families demonstrate that CoVerRL outperforms label-free baselines by 4.7-5.9% on mathematical reasoning benchmarks.<n>Self-verification accuracy improves from around 55% to over 85%, confirming that both capabilities genuinely co-evolve.
arXiv Detail & Related papers (2026-03-18T14:38:55Z)
SeBERTis: A Framework for Producing Classifiers of Security-Related Issue Reports [8.545800179148442]
SEBERTIS is a framework to train Deep Neural Networks (DNNs) as classifiers independent of lexical cues.<n>Our framework achieves a 0.9880 F1-score in detecting security-related issues of a curated corpus of 10,000 GitHub issue reports.
arXiv Detail & Related papers (2025-12-17T01:23:11Z)
Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models [0.7910367295422812]
Large language models (LLMs) make mistakes and can explore unproductive reasoning paths.<n>Self-correction capability is essential for deploying LLMs in safety-critical applications.<n>We uncover a systematic failure: LLMs cannot correct errors in their own outputs while successfully correcting identical errors from external sources.
arXiv Detail & Related papers (2025-07-03T16:41:30Z)
Verbalized Confidence Triggers Self-Verification: Emergent Behavior Without Explicit Reasoning Supervision [12.287123198288079]
Uncertainty calibration is essential for the safe deployment of large language models (LLMs)<n>We find that supervised fine-tuning with scalar confidence labels alone suffices to elicit self-verification behavior of language models.<n>We propose a simple rethinking method that boosts performance via test-time scaling based on calibrated uncertainty.
arXiv Detail & Related papers (2025-06-04T08:56:24Z)
Can Large Reasoning Models Self-Train? [51.0277533541394]
We use majority voting as a simple self-feedback mechanism to study whether self-training can be sustained within reinforcement learning.<n>We find that this basic approach improves not only the model's reasoning performance, but also its capability of generating better quality feedback for the next RL iteration.<n>Yet our analysis also reveals a critical limitation of such a self-training paradigm - prolonged RL with self-reward leads to reward hacking, resulting in sudden and complete performance collapse.
arXiv Detail & Related papers (2025-05-27T17:16:00Z)
SEAL: Steerable Reasoning Calibration of Large Language Models for Free [58.931194824519935]
Large Language Models (LLMs) have demonstrated compelling capabilities for complex reasoning tasks via the extended chain-of-thought (CoT) reasoning mechanism.<n>Recent studies reveal substantial redundancy in the CoT reasoning traces, which negatively impacts model performance.<n>We introduce SEAL, a training-free approach that seamlessly calibrates the CoT process, improving accuracy while demonstrating significant efficiency gains.
arXiv Detail & Related papers (2025-04-07T02:42:07Z)
How Robust are Randomized Smoothing based Defenses to Data Poisoning? [66.80663779176979]
We present a previously unrecognized threat to robust machine learning models that highlights the importance of training-data quality. We propose a novel bilevel optimization-based data poisoning attack that degrades the robustness guarantees of certifiably robust classifiers. Our attack is effective even when the victim trains the models from scratch using state-of-the-art robust training methods.
arXiv Detail & Related papers (2020-12-02T15:30:21Z)
Uncertainty-aware Self-training for Text Classification with Few Labels [54.13279574908808]
We study self-training as one of the earliest semi-supervised learning approaches to reduce the annotation bottleneck. We propose an approach to improve self-training by incorporating uncertainty estimates of the underlying neural network. We show our methods leveraging only 20-30 labeled samples per class for each task for training and for validation can perform within 3% of fully supervised pre-trained language models.
arXiv Detail & Related papers (2020-06-27T08:13:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.