Beyond Single-Vulnerability Evaluation: Closing the Engineering Decision Gap Between C Retrofits and Native Safety

• URL: http://arxiv.org/abs/2608.15486v1
• Date: Sun, 16 Aug 2026 02:29:15 GMT
• Title: Beyond Single-Vulnerability Evaluation: Closing the Engineering Decision Gap Between C Retrofits and Native Safety
• Abstract summary: This paper presents a new evaluation paradigm that benchmarks native memory-safe languages like Rust and Go against compounded C retrofits.<n>Using standardized cross-language tasks, we evaluate the performance and protection tradeoffs of state-of-the-art mechanisms when deployed in combination.
• Score: 2.6022192395158568
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While decades of research have produced numerous retrofitted memory-safety protections for C, these mechanisms are almost exclusively evaluated in isolation, targeting specific vulnerability classes. This siloed evaluation paradigm leaves practitioners without a clear understanding of the cumulative performance costs, interoperability conflicts, and protection gaps that arise when layering defenses to achieve comprehensive safety. This paper presents a new evaluation paradigm that benchmarks natively memory-safe languages like Rust and Go against compounded C retrofits. Using standardized cross-language tasks, we evaluate the performance and protection tradeoffs of state-of-the-art mechanisms when deployed in combination. Our results demonstrate that layered C defenses incur compounding and workload-dependent performance penalties, can suffer from fundamental architectural incompatibilities, and fall short of the protection scope provided by native memory-safe languages. These findings expose a critical engineering decision gap where the true cost of backporting safety to C remains hidden from practitioners. We argue for a fundamental shift in memory-safety research: moving away from isolated evaluation toward holistic, comparative frameworks that inform the high-stakes choice between retrofitting legacy codebases and migrating to modern, safe languages.

Related papers

• Rust for Secure Backend Development: A Critical Review and Extended Vulnerability Comparison with Node.js and Django [2.857200736882273]
  Rust programming language is widely credited with eliminating entire classes of memory-safety and contexts vulnerabilities.<n>This paper presents a critical review of prior work on Rust's security posture in industrial settings.<n>We show that Rust's compile-time guarantees dominate at the systems layer, while managed backend frameworks offer stronger built-in defenses at the application layer.
  arXiv  Detail & Related papers  (2026-08-23T22:15:10Z)
• COP-Q: Safety-First Reinforcement Learning for Robot Control via Cholesky-Ordered Projection [13.702950888176213]
  In off-policy safe reinforcement learning, reward and safety Q-values are commonly learned by separate critic ensembles.<n>This objective-wise treatment neglects inter-objective correlation and can lead to overly conservative value estimates.<n>We propose Cholesky-Ordered Projection Q-learning (COP-Q), a safety-first method that incorporates inter-objective covariance into vector-valued Q-value estimation.
  arXiv  Detail & Related papers  (2026-06-03T11:30:10Z)
• False Security Confidence in Benign LLM Code Generation [0.0]
  This note introduces a complementary but distinct framing: False Security Confidence (FSC)<n>FSC studies the same surface phenomenon from a measurement-first perspective in ordinary, non-attack-framed generation tasks.<n>We formalize FSC rate as the prevalence of security failure within the set of functionally correct outputs.
  arXiv  Detail & Related papers  (2026-04-18T14:52:26Z)
• MemEvoBench: Benchmarking Memory MisEvolution in LLM Agents [78.95081012334116]
  Large Language Models (LLMs) with persistent memory enhances interaction continuity and personalization but introduces new safety risks.<n> contaminated or biased memory accumulation can trigger abnormal agent behaviors.<n>MemeEvoBench is the first benchmark evaluating memory safety in LLM agents against adversarial memory injection, noisy tool outputs, and biased feedback.
  arXiv  Detail & Related papers  (2026-04-17T07:29:52Z)
• SecPI: Secure Code Generation with Reasoning Models via Security Reasoning Internalization [50.71047638695205]
  Reasoning language models (RLMs) are increasingly used in programming.<n>Yet, even state-of-the-art RLMs frequently introduce critical security vulnerabilities in generated code.<n>We present SecPI, a fine-tuning pipeline that teaches RLMs to internalize structured security reasoning.
  arXiv  Detail & Related papers  (2026-04-04T04:29:11Z)
• CIBER: A Comprehensive Benchmark for Security Evaluation of Code Interpreter Agents [27.35968236632966]
  LLM-based code interpreter agents are increasingly deployed in critical situations.<n>Existing benchmarks fail to capture the security risks arising from dynamic code execution, tool interactions, and multi-turn context.<n>We introduce CIBER, an automated benchmark that combines dynamic attack generation, isolated secure sandboxing, and state-aware evaluation.
  arXiv  Detail & Related papers  (2026-02-23T06:41:41Z)
• RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories [58.32028251925354]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in code generation, but their proficiency in producing secure code remains a critical, under-explored area.<n>We introduce RealSec-bench, a new benchmark for secure code generation meticulously constructed from real-world, high-risk Java repositories.
  arXiv  Detail & Related papers  (2026-01-30T08:29:01Z)
• Attributing and Exploiting Safety Vectors through Global Optimization in Large Language Models [50.91504059485288]
  We propose a framework that identifies safety-critical attention heads through global optimization over all heads simultaneously.<n>We develop a novel inference-time white-box jailbreak method that exploits the identified safety vectors through activation repatching.
  arXiv  Detail & Related papers  (2026-01-22T09:32:43Z)
• ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoning [64.32925552574115]
  ARMOR is a large language model that analyzes jailbreak strategies and extracts the core intent.<n> ARMOR achieves state-of-the-art safety performance, with an average harmful rate of 0.002 and an attack success rate of 0.06 against advanced optimization-based jailbreaks.
  arXiv  Detail & Related papers  (2025-07-14T09:05:54Z)
• Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking [54.10710423370126]
  We propose Reasoning-to-Defend (R2D), a training paradigm that integrates a safety-aware reasoning mechanism into Large Language Models' generation process.<n>CPO enhances the model's perception of the safety status of given dialogues.<n>Experiments demonstrate that R2D effectively mitigates various attacks and improves overall safety, while maintaining the original performances.
  arXiv  Detail & Related papers  (2025-02-18T15:48:46Z)
• Evaluating Defences against Unsafe Feedback in RLHF [26.872318173182414]
  This paper looks at learning from unsafe feedback with reinforcement learning.<n>We find that safety-aligned LLMs easily explore unsafe action spaces via generating harmful text.<n>In order to protect against this vulnerability, we adapt a number of both "implict" and "explicit" harmful fine-tuning defences.
  arXiv  Detail & Related papers  (2024-09-19T17:10:34Z)
• The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness [56.174255970895466]
  Large Language Models (LLMs) play an increasingly pivotal role in natural language processing applications. This paper presents Safety and Over-Defensiveness Evaluation (SODE) benchmark.
  arXiv  Detail & Related papers  (2023-12-30T17:37:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.