False Security Confidence in Benign LLM Code Generation

• URL: http://arxiv.org/abs/2604.17014v2
• Date: Tue, 21 Apr 2026 15:54:52 GMT
• Title: False Security Confidence in Benign LLM Code Generation
• Abstract summary: This note introduces a complementary but distinct framing: False Security Confidence (FSC)<n>FSC studies the same surface phenomenon from a measurement-first perspective in ordinary, non-attack-framed generation tasks.<n>We formalize FSC rate as the prevalence of security failure within the set of functionally correct outputs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prior work has demonstrated that functionally correct yet vulnerable outputs arise systematically in threat-oriented settings, where adversarial or implicit channels are used to induce security failures in code agents and automated patching workflows. This note introduces a complementary but distinct framing: False Security Confidence (FSC), which studies the same surface phenomenon from a measurement-first perspective in ordinary, non-attack-framed generation tasks. Our interest is not in whether attacks can produce such outputs, but in how frequently and in what forms they appear absent explicit attack pressure, and whether conventional functional evaluation reliably detects them. We formalize FSC rate as the prevalence of security failure within the set of functionally correct outputs, distinguish it from prior joint functional-security metrics such as SAFE and outcome-driven evaluation frameworks such as CWEval, define a three-ecosystem task view for studying how FSC manifests across general-purpose programming, deployment-context tasks, and security-explicit programming, and identify FSC-hard as a practically important refinement layer in which static analyzers miss vulnerabilities that remain dynamically triggerable. This technical report is intentionally scoped as a framework statement rather than a full empirical paper: its purpose is to establish terminology, measurement boundaries, and study design commitments for subsequent large-scale evaluation.

Related papers

• AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation [48.55515767840701]
  "AI slop" is artifacts, hallucinated vulnerabilities, plausible but incorrect patches, and semantically repackaged bug reports.<n>This paper surveys the empirical evidence, identifies a unifying mechanism, and traces a path toward trustworthy triage.
  arXiv  Detail & Related papers  (2026-08-26T11:48:38Z)
• Conformal Changepoint Localization and Root Cause Analysis with Corrupted Observations [55.76952683833966]
  This paper proposes weighted CONCH (W-CONCH) and weighted CROC (W-CROC) to reduce confidence set size when data may be corrupted.<n>Experiments on image-based and real-world changepoint and root-cause benchmarks show that uncertainty-based weighting substantially reduces confidence set size.
  arXiv  Detail & Related papers  (2026-07-29T05:16:59Z)
• SeClaw: Spec-Driven Security Task Synthesis for Evaluating Autonomous Agents [87.26967184869198]
  SeClaw is a framework that combines specification-driven security task synthesis with execution-based security evaluation for Autonomous agents.<n> benchmark covers risks arising from resources, user tasks, environments, and intrinsic agent behaviors.
  arXiv  Detail & Related papers  (2026-06-01T14:23:42Z)
• SecPI: Secure Code Generation with Reasoning Models via Security Reasoning Internalization [50.71047638695205]
  Reasoning language models (RLMs) are increasingly used in programming.<n>Yet, even state-of-the-art RLMs frequently introduce critical security vulnerabilities in generated code.<n>We present SecPI, a fine-tuning pipeline that teaches RLMs to internalize structured security reasoning.
  arXiv  Detail & Related papers  (2026-04-04T04:29:11Z)
• SFCoT: Safer Chain-of-Thought via Active Safety Evaluation and Calibration [9.8659606914942]
  This paper proposes a SaFer Chain-of-Thought (SFCoT) framework, which proactively evaluates and calibrates potentially unsafe reasoning steps in real time.<n> Experimental results demonstrate that SFCoT reduces the attack success rate from $58.97%$ to $12.31%$.
  arXiv  Detail & Related papers  (2026-03-16T15:13:21Z)
• Embedding Poisoning: Bypassing Safety Alignment via Embedding Semantic Shift [23.0914017433021]
  This work identifies a novel class of deployment phase attacks that exploit a vulnerability by injecting imperceptible perturbations directly into the embedding layer outputs without modifying model weights or input text.<n>We propose Search based Embedding Poisoning, a practical, model agnostic framework that introduces carefully optimized perturbations into embeddings associated with high risk tokens.
  arXiv  Detail & Related papers  (2025-09-08T05:00:58Z)
• A Dynamical Systems Framework for Reinforcement Learning Safety and Robustness Verification [1.104960878651584]
  This paper introduces a novel framework that addresses the lack of formal methods for verifying the robustness and safety of learned policies.<n>By leveraging tools from dynamical systems theory, we identify and visualize Lagrangian Coherent Structures (LCS) that act as the hidden "skeleton" governing the system's behavior.<n>We show that this framework provides a comprehensive and interpretable assessment of policy behavior, successfully identifying critical flaws in policies that appear successful based on reward alone.
  arXiv  Detail & Related papers  (2025-08-21T14:00:26Z)
• Beyond Algorithmic Proofs: Towards Implementation-Level Provable Security [1.338174941551702]
  We present Implementation-Level Provable Security, a new paradigm that defines security in terms of structurally verifiable resilience against real-world attack surfaces during deployment.<n>We present SEER (Secure and Efficient Encryption-based Erasure via Ransomware), a file destruction system that repurposes and reinforces the encryption core of Babuk ransomware.
  arXiv  Detail & Related papers  (2025-08-02T01:58:06Z)
• Preliminary Investigation into Uncertainty-Aware Attack Stage Classification [81.28215542218724]
  This work addresses the problem of attack stage inference under uncertainty.<n>We propose a classification approach based on Evidential Deep Learning (EDL), which models predictive uncertainty by outputting parameters of a Dirichlet distribution over possible stages.<n>Preliminary experiments in a simulated environment demonstrate that the proposed model can accurately infer the stage of an attack with confidence.
  arXiv  Detail & Related papers  (2025-08-01T06:58:00Z)
• Advancing Embodied Agent Security: From Safety Benchmarks to Input Moderation [52.83870601473094]
  Embodied agents exhibit immense potential across a multitude of domains.<n>Existing research predominantly concentrates on the security of general large language models.<n>This paper introduces a novel input moderation framework, meticulously designed to safeguard embodied agents.
  arXiv  Detail & Related papers  (2025-04-22T08:34:35Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.