Poster: Rethinking Security in LLM Code Generation through Real-World Risk Scenarios

• URL: http://arxiv.org/abs/2607.23088v1
• Date: Sat, 25 Jul 2026 07:46:13 GMT
• Title: Poster: Rethinking Security in LLM Code Generation through Real-World Risk Scenarios
• Abstract summary: Large Language Models (LLMs) are widely used for code generation, yet their security behavior in realistic development remains underexplored.<n>We identify three representative risk scenarios that commonly lead to security vulnerabilities in LLM-generated code.<n>We demonstrate that security-aware prompting can substantially mitigate these risks, achieving up to 45% improvement.
• Score: 26.44803027861766
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) are widely used for code generation, yet their security behavior in realistic development workflows remains underexplored. Existing benchmarks often rely on explicitly specified security requirements, failing to capture real-world scenarios where prompts are frequently ambiguous or incomplete. In this paper, we adopt a developer-centric perspective and identify three representative risk scenarios that commonly lead to security vulnerabilities in LLM-generated code: Ambiguous Requirements, Under-Specified Operational Context, and Security--Functionality Conflict. Based on these scenarios, we construct a large-scale benchmark comprising 2,700 test cases, enabling fine-grained evaluation of LLM security under realistic conditions. Extensive evaluation of eight state-of-the-art LLMs reveals that all models exhibit average vulnerability rates exceeding 56\% across risk scenarios. We further demonstrate that security-aware prompting can substantially mitigate these risks, achieving up to 45\% improvement.

Related papers

• VESTA: A Fully Automated Scenario Generation and Safety Evaluation Framework for LLM Agents [7.980753712857791]
  We introduce VESTA, a fully automated scenario generation and safety evaluation framework for large language models (LLMs)<n> VESTA instantiaes abstract and diverse safety risks in real-world task execution into 1,072 measurable evaluation scenarios.<n>The results show that current agents still face substantial behavioral safety risks during task execution, with an average ASR of 47.1% and several models exceeding 70%.
  arXiv  Detail & Related papers  (2026-06-07T09:23:38Z)
• A New Framework for Cybersecurity Refusals in AI Agents [52.94784168139071]
  We present the first framework for establishing refusal boundaries in offensive security contexts.<n>We apply this framework to assess how current LLM-powered agents adhere to appropriate refusal boundaries across a range of web-based offensive security scenarios.
  arXiv  Detail & Related papers  (2026-05-31T15:39:39Z)
• Security Assessment and Mitigation Strategies for Large Language Models: A Comprehensive Defensive Framework [0.6015898117103067]
  Large Language Models increasingly power critical infrastructure from healthcare to finance, yet their vulnerability to adversarial manipulation threatens system integrity and user safety.<n>This research addresses this gap by establishing a standardized vulnerability assessment framework and developing a multi-layered defensive system to protect against identified threats.
  arXiv  Detail & Related papers  (2026-03-17T20:32:06Z)
• A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code [49.009041488527544]
  A.S.E is a repository-level evaluation benchmark for assessing the security of AI-generated code.<n>Current large language models (LLMs) still struggle with secure coding.<n>A larger reasoning budget does not necessarily lead to better code generation.
  arXiv  Detail & Related papers  (2025-08-25T15:11:11Z)
• OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety [58.201189860217724]
  We introduce OpenAgentSafety, a comprehensive framework for evaluating agent behavior across eight critical risk categories.<n>Unlike prior work, our framework evaluates agents that interact with real tools, including web browsers, code execution environments, file systems, bash shells, and messaging platforms.<n>It combines rule-based analysis with LLM-as-judge assessments to detect both overt and subtle unsafe behaviors.
  arXiv  Detail & Related papers  (2025-07-08T16:18:54Z)
• RAS-Eval: A Comprehensive Benchmark for Security Evaluation of LLM Agents in Real-World Environments [5.3532199508293745]
  RAS-Eval is a comprehensive security benchmark supporting both simulated and real-world tool execution.<n>We evaluate 6 state-of-the-art Large language model (LLM) agents across diverse scenarios.<n>Attacks reduced agent task completion rates (TCR) by 36.78% on average and achieved an 85.65% success rate in academic settings.
  arXiv  Detail & Related papers  (2025-06-18T08:30:36Z)
• ROSE: Toward Reality-Oriented Safety Evaluation of Large Language Models [60.28667314609623]
  Large Language Models (LLMs) are increasingly deployed as black-box components in real-world applications.<n>We propose Reality-Oriented Safety Evaluation (ROSE), a novel framework that uses multi-objective reinforcement learning to fine-tune an adversarial LLM.
  arXiv  Detail & Related papers  (2025-06-17T10:55:17Z)
• SafeGenBench: A Benchmark Framework for Security Vulnerability Detection in LLM-Generated Code [7.209766132478914]
  We introduce SafeGenBench, a benchmark specifically designed to assess the security of LLM-generated code.<n>The dataset encompasses a wide range of common software development scenarios and vulnerability types.<n>Through the empirical evaluation of state-of-the-art LLMs on SafeGenBench, we reveal notable deficiencies in their ability to produce vulnerability-free code.
  arXiv  Detail & Related papers  (2025-06-06T02:48:02Z)
• SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models [75.67623347512368]
  We propose toolns, a comprehensive framework designed for conducting safety evaluations of MLLMs. Our framework consists of a comprehensive harmful query dataset and an automated evaluation protocol. Based on our framework, we conducted large-scale experiments on 15 widely-used open-source MLLMs and 6 commercial MLLMs.
  arXiv  Detail & Related papers  (2024-10-24T17:14:40Z)
• Safety Assessment of Chinese Large Language Models [51.83369778259149]
  Large language models (LLMs) may generate insulting and discriminatory content, reflect incorrect social values, and may be used for malicious purposes. To promote the deployment of safe, responsible, and ethical AI, we release SafetyPrompts including 100k augmented prompts and responses by LLMs.
  arXiv  Detail & Related papers  (2023-04-20T16:27:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.