Specification-Driven Development Benchmark: Security Knowledge Transition

• URL: http://arxiv.org/abs/2606.00167v1
• Date: Fri, 29 May 2026 12:36:48 GMT
• Title: Specification-Driven Development Benchmark: Security Knowledge Transition
• Abstract summary: This paper proposes a security knowledge operationalization approach for AI-assisted specification-driven development.<n>A Multilayer Specification Security Model represents security knowledge through traceable relations between system entities.<n>A Security Knowledge Transition Method transforms business and technical specifications into a validated security-enriched generation contract.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI-assisted software development is shifting from isolated code completion toward specification-driven generation, where business requirements, technical specifications, and acceptance criteria become operational input for LLM-based development agents. This shift creates a security problem: functional behavior is described explicitly, while security behavior remains implicit, generic, or postponed to post-generation review, causing generated systems to satisfy visible functional requirements while failing to preserve authorization rules, ownership boundaries, input validation, token rejection, sensitive data handling, and abuse-case semantics. This paper proposes a security knowledge operationalization approach for AI-assisted specification-driven development, combining two contributions: a Multilayer Specification Security Model that represents security knowledge through traceable relations between system entities, threats, risks, requirements, implementation rules, controls, verification scenarios, and evidence; and a Security Knowledge Transition Method that transforms business and technical specifications into a validated security-enriched generation contract. We evaluate the approach through two empirical studies: a hidden-oracle study assessing whether an LLM-based pipeline can derive a structured security model from system context, and a backend generation study under three conditions: no explicit security requirements, ASVS-conditioned generation, and Multilayer Security Model conditioning. Evaluated against a hidden 221-test black-box API suite, modal failures decreased from 50 in the baseline to 42 with ASVS and 36 with the Multilayer Security Model, with the strongest improvements in application-specific categories such as business logic and admin safety.

Related papers

• CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wild [49.44069582581181]
  CyberFactory transforms public vulnerability artifacts, including CVEs from the wild, into executable and verifiable task instances.<n>It uses a reusable vulnerability-analysis skill to guide the teacher through source inspection, problem solving with domain prior, and evidence-based validation.<n>On CyberGym, modelname reaches 52.4% Pass@1 under a one-hour budget, improving over its Qwen3.5 base model by +22.8 points.
  arXiv  Detail & Related papers  (2026-08-24T12:30:10Z)
• The Illusion of Secure LLM Code: Closing the Security Gap via Iterative Reprompting [0.6066442015301664]
  This paper evaluates the security architecture of authentication systems generated by five prominent AI coding assistants.<n> Empirical results demonstrate that code generated from functional or generically secure prompts consistently omits critical protections.<n>Ultimately, this study proves that current AI coding assistants do not produce secure-by-default applications.
  arXiv  Detail & Related papers  (2026-07-26T15:16:44Z)
• SeClaw: Spec-Driven Security Task Synthesis for Evaluating Autonomous Agents [87.26967184869198]
  SeClaw is a framework that combines specification-driven security task synthesis with execution-based security evaluation for Autonomous agents.<n> benchmark covers risks arising from resources, user tasks, environments, and intrinsic agent behaviors.
  arXiv  Detail & Related papers  (2026-06-01T14:23:42Z)
• SecPI: Secure Code Generation with Reasoning Models via Security Reasoning Internalization [50.71047638695205]
  Reasoning language models (RLMs) are increasingly used in programming.<n>Yet, even state-of-the-art RLMs frequently introduce critical security vulnerabilities in generated code.<n>We present SecPI, a fine-tuning pipeline that teaches RLMs to internalize structured security reasoning.
  arXiv  Detail & Related papers  (2026-04-04T04:29:11Z)
• Security Considerations for Artificial Intelligence Agents [7.055090485438426]
  This article details Perplexity's observations and recommendations concerning the security of frontier AI agents.<n>We map attack surfaces across tools, connectors, hosting boundaries, and multi-agent coordination.<n>We identify standards and research gaps, guidance for secure multi-agent system design aligned with NIST risk management principles.
  arXiv  Detail & Related papers  (2026-03-12T17:49:39Z)
• Inference-Time Safety For Code LLMs Via Retrieval-Augmented Revision [3.983997834693767]
  Large Language Models (LLMs) are increasingly deployed for code generation in high-stakes software development.<n>LLMs cannot readily adapt to newly discovered vulnerabilities or changing security standards without retraining.<n>We present a principled approach to trustworthy code generation by design that operates as an inference-time safety mechanism.
  arXiv  Detail & Related papers  (2026-03-02T06:06:34Z)
• Constitutional Spec-Driven Development: Enforcing Security by Construction in AI-Assisted Code Generation [0.0]
  AI-assisted "vibe coding" enables rapid software development but introduces significant security risks.<n>We present Constitutional Spec-Driven Development, a methodology that embeds non-negotiable security principles into the specification.<n>We show that constitutional constraints reduce security defects by 73% compared to unconstrained AI generation.
  arXiv  Detail & Related papers  (2026-01-31T19:08:16Z)
• Towards Verifiably Safe Tool Use for LLM Agents [53.55621104327779]
  Large language model (LLM)-based AI agents extend capabilities by enabling access to tools such as data sources, APIs, search engines, code sandboxes, and even other agents.<n>LLMs may invoke unintended tool interactions and introduce risks, such as leaking sensitive data or overwriting critical records.<n>Current approaches to mitigate these risks, such as model-based safeguards, enhance agents' reliability but cannot guarantee system safety.
  arXiv  Detail & Related papers  (2026-01-12T21:31:38Z)
• An Ontology-Based Approach to Security Risk Identification of Container Deployments in OT Contexts [1.826848871278733]
  Security risk identification for OT container deployments is challenged by hybrid IT/OT architectures, fragmented stakeholder knowledge, and continuous system changes.<n>We propose a model-based approach, implemented as the Container Security Risk Ontology (CSRO)<n>CSRO integrates five key domains: adversarial behaviour, contextual assumptions, attack scenarios, risk assessment rules, and container security artefacts.
  arXiv  Detail & Related papers  (2026-01-07T15:20:19Z)
• Aspect-Oriented Programming in Secure Software Development: A Case Study of Security Aspects in Web Applications [0.0]
  This study investigates the role of Aspect-Oriented Programming (AOP) in enhancing secure software development.<n>We compare AOP-based implementations of security features including authentication, authorization, input validation, encryption, logging, and session management.<n>The findings demonstrate that AOP enhances modularity, reusability, and maintainability of security mechanisms, while introducing only minimal performance overhead.
  arXiv  Detail & Related papers  (2025-09-09T07:12:55Z)
• Secure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Security [63.41350337821108]
  We propose Secure Tug-of-War (SecTOW) to enhance the security of multimodal large language models (MLLMs)<n>SecTOW consists of two modules: a defender and an auxiliary attacker, both trained iteratively using reinforcement learning (GRPO)<n>We show that SecTOW significantly improves security while preserving general performance.
  arXiv  Detail & Related papers  (2025-07-29T17:39:48Z)
• Advancing Neural Network Verification through Hierarchical Safety Abstract Interpretation [52.626086874715284]
  We introduce a novel problem formulation called Abstract DNN-Verification, which verifies a hierarchical structure of unsafe outputs.<n>By leveraging abstract interpretation and reasoning about output reachable sets, our approach enables assessing multiple safety levels during the formal verification process.<n>Our contributions include a theoretical exploration of the relationship between our novel abstract safety formulation and existing approaches.
  arXiv  Detail & Related papers  (2025-05-08T13:29:46Z)
• SeCodePLT: A Unified Platform for Evaluating the Security of Code GenAI [58.29510889419971]
  Existing benchmarks for evaluating the security risks and capabilities of code-generating large language models (LLMs) face several key limitations.<n>We introduce a general and scalable benchmark construction framework that begins with manually validated, high-quality seed examples and expands them via targeted mutations.<n>Applying this framework to Python, C/C++, and Java, we build SeCodePLT, a dataset of more than 5.9k samples spanning 44 CWE-based risk categories and three security capabilities.
  arXiv  Detail & Related papers  (2024-10-14T21:17:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.