Residual Risk Analysis in Benign Code: How Far Are We? A Multi-Model Semantic and Structural Similarity Approach

• URL: http://arxiv.org/abs/2604.21051v1
• Date: Wed, 22 Apr 2026 19:51:31 GMT
• Title: Residual Risk Analysis in Benign Code: How Far Are We? A Multi-Model Semantic and Structural Similarity Approach
• Abstract summary: Existing vulnerability benchmarks often treat patched functions as inherently benign, overlooking the possibility of residual security risks.<n>We propose Residual Risk Scoring (RRS), a unified framework that integrates embedding-based semantic similarity, localized AST-based structural similarity, and cross-model agreement to estimate residual risk in code.<n>Our analysis shows that benign functions often remain highly similar to their vulnerable counterparts both semantically and structurally, indicating potential persistence of residual risk.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Software security relies on effective vulnerability detection and patching, yet determining whether a patch fully eliminates risk remains an underexplored challenge. Existing vulnerability benchmarks often treat patched functions as inherently benign, overlooking the possibility of residual security risks. In this work, we analyze vulnerable-benign function pairs from the PrimeVul, a benchmark dataset using multiple code language models (Code LMs) to capture semantic similarity, complemented by Tree-sitter-based abstract syntax tree (AST) analysis for structural similarity. Building on these signals, we propose Residual Risk Scoring (RRS), a unified framework that integrates embedding-based semantic similarity, localized AST-based structural similarity, and cross-model agreement to estimate residual risk in code. Our analysis shows that benign functions often remain highly similar to their vulnerable counterparts both semantically and structurally, indicating potential persistence of residual risk. We further find that approximately $61\%$ of high-RRS code pairs exhibit $13$ distinct categories of residual issues (e.g., null pointer dereferences, unsafe memory allocation), validated using state-of-the-art static analysis tools including Cppcheck, Clang-Tidy, and Facebook-Infer. These results demonstrate that code-level similarity provides a practical signal for prioritizing post-patch inspection, enabling more reliable and scalable security assessment in real-world open-source software pipelines.

Related papers

• AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation [48.55515767840701]
  "AI slop" is artifacts, hallucinated vulnerabilities, plausible but incorrect patches, and semantically repackaged bug reports.<n>This paper surveys the empirical evidence, identifies a unifying mechanism, and traces a path toward trustworthy triage.
  arXiv  Detail & Related papers  (2026-08-26T11:48:38Z)
• DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detection [16.007199928732614]
  We present DREA, a hypothesis-driven framework for repository-level vulnerability detection.<n>Goal-directed context acquisition is the primary source of detection improvement in this design.<n>To support evaluation, we construct RepoPairBench, a repository-grounded benchmark of validated Python vulnerability-fix pairs from real-world projects.
  arXiv  Detail & Related papers  (2026-07-15T04:49:05Z)
• The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Models [49.67749928542536]
  Safety alignment of text-to-image (T2I) diffusion models aims to suppress harmful generations while preserving utility on benign prompts.<n>Recent methods often appear to deliver high safety with high utility, but this conclusion rests largely on coarse global utility metrics.<n>We show that when utility is measured with structured evaluation, this illusion breaks: onA (Text-to-Image Faithfulness evaluation with Question Answering)<n>We propose StructureAware Geometric Regularization (SAGE), a safety alignment objective that explicitly preserves embedding spread and inter-prompt relational structure.
  arXiv  Detail & Related papers  (2026-07-01T04:00:27Z)
• From Sparse Features to Trustworthy Proxies: Certifying SAE-Based Interpretability [53.681678236115836]
  We study when can an SAE-based explanation be treated as a faithful view of an underlying frozen LM.<n>Our framework derives an upper bound on the base model's expected risk using four measurable quantities.<n>We show that the bound becomes non-vacuous on GPT-2 Small, Gemma-2B, and Llama-3-8B at practical sample sizes.
  arXiv  Detail & Related papers  (2026-06-16T18:28:23Z)
• Quantifiable Uncertainty: A Stochastic Consensus Multi-Agent RAG Framework for Robust Malware Detection [1.2183405753834562]
  We present MAGMA, a framework that decouples malware analysis into semantic code retrieval and probabilistic verification.<n>We show that MAGMA achieves a 98.4% detection rate, substantially exceeding existing solutions.
  arXiv  Detail & Related papers  (2026-05-08T18:46:24Z)
• From Agent Failure Paths to Quantified Residual Risk: A Compositional Framework for Resilient Agentic AI [8.947753534117298]
  Agentic AI is crossing trust boundaries faster than current risk models can represent.<n>We propose a mechanism-to-magnitude pipeline for resilient agentic and embodied AI.<n>We show the framework on two contrasting scenarios a real-time warehouse robot and a governance-instrumented financial-services agent.
  arXiv  Detail & Related papers  (2026-04-28T23:40:24Z)
• ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis [96.92417622318267]
  ATBench is a trajectory-level benchmark for structured, diverse, and realistic evaluation of agent safety.<n>It organizes agentic risk along three dimensions: risk source, failure mode, and real-world harm.<n>It contains 1,000 trajectories (503 safe and 497 unsafe), averaging 9.01 turns and 3.95k tokens, with 1,954 invoked tools drawn from pools spanning 2,084 available tools.
  arXiv  Detail & Related papers  (2026-04-02T13:26:20Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• VulnAgent-X: A Layered Agentic Framework for Repository-Level Vulnerability Detection [9.51089779418208]
  VulnAgentX is a layered agentic framework integrat- ing lightweight risk screening, bounded context expansion, specialised analysis agents, selective dynamic verification, and evidence fusion into a unified pipeline.<n>Treating vulnerabil- ity detection as a staged, evidence-driven auditing process improves de- tection quality, reduces false positives, and produces interpretable re-sults for repository-level software security analysis.
  arXiv  Detail & Related papers  (2026-03-11T04:57:17Z)
• RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories [58.32028251925354]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in code generation, but their proficiency in producing secure code remains a critical, under-explored area.<n>We introduce RealSec-bench, a new benchmark for secure code generation meticulously constructed from real-world, high-risk Java repositories.
  arXiv  Detail & Related papers  (2026-01-30T08:29:01Z)
• The Semantic Trap: Do Fine-tuned LLMs Learn Vulnerability Root Cause or Just Functional Pattern? [14.472036099680961]
  We propose TrapEval, a comprehensive evaluation framework designed to disentangle vulnerability root cause from functional pattern.<n>We fine-tune five representative state-of-the-art LLMs across three model families and evaluate them under cross-dataset testing, semantic-preservings, and varying degrees of semantic gap measured by CodeBLEU.<n>Our findings serve as a wake-up call: high benchmark scores on traditional datasets may be illusory, masking the model's inability to understand the true causal logic of vulnerabilities.
  arXiv  Detail & Related papers  (2026-01-30T07:19:17Z)
• Cracking IoT Security: Can LLMs Outsmart Static Analysis Tools? [1.8549313085249322]
  This work presents the first comprehensive evaluation of Large Language Models (LLMs) across a multi-category interaction threat taxonomy.<n>We benchmark Llama 3.1 8B, Llama 70B, GPT-4o, Gemini-2.5-Pro, and DeepSeek-R1 across zero-, one-, and two-shot settings.<n>Our findings show that while LLMs exhibit promising semantic understanding, their accuracy degrades significantly for threats requiring cross-rule structural reasoning.
  arXiv  Detail & Related papers  (2026-01-02T04:17:36Z)
• Distributionally Robust Safety Verification of Neural Networks via Worst-Case CVaR [3.0458514384586404]
  This paper builds on Fazlyab's quadratic-constraint (QC) and semidefinite-programming (SDP) framework for neural network verification.<n>The integration broadens input-uncertainty geometry-covering ellipsoids, polytopes, and hyperplanes-and extends applicability to safety-critical domains.
  arXiv  Detail & Related papers  (2025-09-22T07:04:53Z)
• Advancing Neural Network Verification through Hierarchical Safety Abstract Interpretation [52.626086874715284]
  We introduce a novel problem formulation called Abstract DNN-Verification, which verifies a hierarchical structure of unsafe outputs.<n>By leveraging abstract interpretation and reasoning about output reachable sets, our approach enables assessing multiple safety levels during the formal verification process.<n>Our contributions include a theoretical exploration of the relationship between our novel abstract safety formulation and existing approaches.
  arXiv  Detail & Related papers  (2025-05-08T13:29:46Z)
• Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation [74.05906222376608]
  We propose adversarial self-supervision UDA (or ASSUDA) that maximizes the agreement between clean images and their adversarial examples by a contrastive loss in the output space. This paper is rooted in two observations: (i) the robustness of UDA methods in semantic segmentation remains unexplored, which pose a security concern in this field; and (ii) although commonly used self-supervision (e.g., rotation and jigsaw) benefits image tasks such as classification and recognition, they fail to provide the critical supervision signals that could learn discriminative representation for segmentation tasks.
  arXiv  Detail & Related papers  (2021-05-23T01:50:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.