Safety Under Scaffolding: How Evaluation Conditions Shape Measured Safety

• URL: http://arxiv.org/abs/2603.10044v1
• Date: Sun, 08 Mar 2026 01:37:45 GMT
• Title: Safety Under Scaffolding: How Evaluation Conditions Shape Measured Safety
• Abstract summary: We report one of the largest controlled studies of scaffold effects on safety.<n>Map-reduce scaffolding degrades measured safety.<n>Switching from multiple-choice to open-ended format on identical items shifts safety scores by 5-20 percentage points.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Safety benchmarks evaluate language models in isolation, typically using multiple-choice format; production deployments wrap these models in agentic scaffolds that restructure inputs through reasoning traces, critic agents, and delegation pipelines. We report one of the largest controlled studies of scaffold effects on safety (N = 62,808; six frontier models, four deployment configurations), combining pre-registration, assessor blinding, equivalence testing, and specification curve analysis. Map-reduce scaffolding degrades measured safety (NNH = 14), yet two of three scaffold architectures preserve safety within practically meaningful margins. Investigating the map-reduce degradation revealed a deeper measurement problem: switching from multiple-choice to open-ended format on identical items shifts safety scores by 5-20 percentage points, larger than any scaffold effect. Within-format scaffold comparisons are consistent with practical equivalence under our pre-registered +/-2 pp TOST margin, isolating evaluation format rather than scaffold architecture as the operative variable. Model x scaffold interactions span 35 pp in opposing directions (one model degrades by -16.8 pp on sycophancy under map-reduce while another improves by +18.8 pp on the same benchmark), ruling out universal claims about scaffold safety. A generalisability analysis yields G = 0.000: model safety rankings reverse so completely across benchmarks that no composite safety index achieves non-zero reliability, making per-model, per-configuration testing a necessary minimum standard. We release all code, data, and prompts as ScaffoldSafety.

Related papers

• EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents [74.54929751174289]
  EvoSafeHarness is a safety-specific optimization framework that synthesizes a deployable harness for a frozen model in a target domain.<n>It reduces average attack success rate from 45.6% to 10.0% at a 3.3-point utility cost.<n>It also achieves the best score on Agent-SafetyBench for every victim.
  arXiv  Detail & Related papers  (2026-09-05T05:56:41Z)
• SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models [73.04773649437375]
  Multimodal safety moderation requires distinguishing risks arising from visual content, user intent, and assistant behavior.<n>We introduce SafeAtlas-VL, a dataset of 1.5M training instances that places image-, request-, and response-level judgments on a five-level ordered scale.<n>We also construct SafeAtlas-Bench, a set of 5,000 instances for evaluating five-level predictions and continuous risk scores.
  arXiv  Detail & Related papers  (2026-08-29T07:08:01Z)
• Item Response Theory for AI Safety [23.16156306184828]
  Item Response Theory (IRT) is a statistical toolkit for measuring latents from performance on items with inferred psychometric properties.<n>IRT is a ready-made toolkit for reading, reducing, and auditing safety benchmarks, which we recommend frontier labs and evaluators adopt.
  arXiv  Detail & Related papers  (2026-08-05T17:25:27Z)
• Shieldstral [183.97744446582863]
  Shieldstral is a policy-adaptive multimodal safety classifier that matches or outperforms models nearly 7$times$ its size on text safety benchmarks.<n>We present a data construction recipe, covering curation and generation of approximately 54.1M samples and a fine-grained evaluation set to evaluate policy adaptability.
  arXiv  Detail & Related papers  (2026-07-28T15:27:53Z)
• Silent Alarm: A J-Space Protocol for Comparing Danger Recognition Across Models and Quantization Levels [0.0]
  This work proposes JADR, a protocol that measures a model's internal representation through Jacobian space.<n>For every prompt and layer we record the top-k J-space tokens.<n>These are grouped into six behavioral scenario axes and compared between a danger sample based on StrongREJECT and a safe control drawn from XSTest and OKTest.
  arXiv  Detail & Related papers  (2026-07-14T14:08:18Z)
• When Context Flips, Safety Breaks: Diagnosing Brittle Safety in Aligned Language Models [5.03712104249503]
  Safety benchmark scores provide incomplete evidence of deployment readiness.<n> aligned language models often adhere to rigid rules even when a situational update flips which action is safe.<n>We test 12 models across a safety benchmark and two commonsense controls using paired variants where the nominally safe action produces harm.
  arXiv  Detail & Related papers  (2026-05-27T02:09:30Z)
• SCAFFOLD-CEGIS: Preventing Latent Security Degradation in LLM-Driven Iterative Code Refinement [12.69450437027072]
  This paper reveals the iterative refinement paradox: specification drift during multi-objective optimization causes security to degrade gradually over successive iterations.<n>We propose the SCAFFOLD-CEGIS framework that transforms security constraints from implicit prompts into explicit verifiable constraints.
  arXiv  Detail & Related papers  (2026-03-09T15:54:18Z)
• How Does Prefix Matter in Reasoning Model Tuning? [57.69882799751655]
  We fine-tune three R1 series models across three core model capabilities: reasoning (mathematics), coding, safety, and factuality.<n>Results show that prefix-conditioned SFT improves both safety and reasoning performance, yielding up to +6% higher Safe@1 accuracy.
  arXiv  Detail & Related papers  (2026-01-04T18:04:23Z)
• Interpretable Safety Alignment via SAE-Constructed Low-Rank Subspace Adaptation [13.509767769174422]
  Safety alignment is critical for training large language models to refuse harmful requests.<n>Low-Rank Adaptation (LoRA) consistently underperforms full fine-tuning and reinforcement learning on safety benchmarks.<n>We propose SAILS (Safety Alignment via Interpretable Low-rank Subspace) to address this gap.
  arXiv  Detail & Related papers  (2025-12-29T07:39:49Z)
• AI Transparency Atlas: Framework, Scoring, and Real-Time Model Card Evaluation Pipeline [2.1787849426740364]
  We analyzed documentation from five frontier models (Gemini 3, Grok 4.1, Llama 4, GPT-5, and Claude 4.5) and 100 Hugging Face model cards.<n>We developed a weighted transparency framework with 8 sections and 23 subsections that prioritizes safety-critical disclosures.
  arXiv  Detail & Related papers  (2025-12-13T19:48:44Z)
• Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMs [6.301699616285567]
  Fine-tuning techniques may exert distinct effects on safety and fairness.<n> adapter-based approaches tend to improve safety scores and are the least disruptive to fairness.<n>Prompt-Tuning and P-Tuning generally reduce safety and cause larger fairness regressions.
  arXiv  Detail & Related papers  (2025-11-01T03:29:56Z)
• VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safety [3.1109025622085693]
  We present Vision Language Safety Understanding, a comprehensive framework to evaluate multimodal safety.<n>Our evaluation of eleven state-of-the-art models reveals systematic joint understanding failures.<n>Our framework exposes weaknesses in joint image-text understanding and alignment gaps in current models.
  arXiv  Detail & Related papers  (2025-10-21T01:30:31Z)
• SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Models [66.71948519280669]
  Multimodal Large Reasoning Models (MLRMs) demonstrate impressive crossmodal reasoning but often amplify safety risks under adversarial prompts.<n> Existing defenses mainly act at the output level and do not constrain the reasoning process, leaving models to implicit risks.<n>We propose SaFeR-VLM, which integrates four components and supports dynamic and interpretable safety decisions beyond surface-level filtering.
  arXiv  Detail & Related papers  (2025-10-08T10:39:12Z)
• UnsafeChain: Enhancing Reasoning Model Safety via Hard Cases [57.69882799751655]
  We release UnsafeChain, a safety alignment dataset constructed from hard prompts with diverse sources.<n>We fine-tune three large reasoning models (LRMs) and compare them against recent SafeChain and STAR-1.<n>UnsafeChain consistently outperforms prior datasets, with even a 1K subset matching or surpassing baseline performance.
  arXiv  Detail & Related papers  (2025-07-29T10:08:52Z)
• What Makes and Breaks Safety Fine-tuning? A Mechanistic Study [64.9691741899956]
  Safety fine-tuning helps align Large Language Models (LLMs) with human preferences for their safe deployment. We design a synthetic data generation framework that captures salient aspects of an unsafe input. Using this, we investigate three well-known safety fine-tuning methods.
  arXiv  Detail & Related papers  (2024-07-14T16:12:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.