Acceptance Cards:A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims

• URL: http://arxiv.org/abs/2605.10575v1
• Date: Mon, 11 May 2026 13:48:42 GMT
• Title: Acceptance Cards:A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims
• Abstract summary: We introduce Acceptance Cards: an evaluation protocol, a documentation object, an executable audit package, and a claim-specific evidential standard for safe fine-tuning defense claims.<n>The protocol checks statistical reliability, fresh semantic generalization, mechanism alignment, and cross-task transfer before treating a gap reduction as a full-card pass.<n>This is a narrow installed-gap audit on one model family, not a global judgment of SafeLoRA's effectiveness.
• Score: 0.6882042556551609
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Safe fine-tuning defenses are often endorsed on the basis of a held-out gap reduction, but the same reduction can come from sampling noise, subject artifacts, capability loss, or a mechanism that does not transfer. We introduce Acceptance Cards: an evaluation protocol, a documentation object, an executable audit package, and a claim-specific evidential standard for safe fine-tuning defense claims. The protocol checks statistical reliability, fresh semantic generalization, mechanism alignment, and cross-task transfer before treating a gap reduction as a full-card pass. Re-scored under this installed-gap protocol, SafeLoRA fails the full-card pass on Gemma-2-2B-it: under strict mechanism-class coding it fails all four diagnostics, and under a permissive shrinkage relabel it still fails three of four. This is a narrow installed-gap audit on one model family, not a global judgment of SafeLoRA's effectiveness. In a 46-cell audit, no cell satisfies the strict conjunction. The closest family is a near miss that passes reliability and mechanism checks where the required data are available, but fails the fresh-subject threshold, lacks a strict transfer pass, and carries a measurable deployment-accuracy cost.

Related papers

• A distribution-free certification framework for trustworthy crash-severity prediction [1.089614199781423]
  We develop a certification layer that wraps any severity model unmodified.<n>On 5.2 million Texas records across seven base models spanning four decades, the layer attaches identical validity.<n>The framework is released as an open-source package with theorem-level tests.
  arXiv  Detail & Related papers  (2026-09-10T14:20:08Z)
• Explicit State Elicitation Is Not Enough: A Controlled Audit of Memory-Policy Classification [11.991995412984503]
  We use this setting to develop an empirical audit protocol for structured intermediate outputs.<n>A 480-example synthetic development set initially suggested large gains from a state-structured prompt bundle.<n>On this set, exposing the four state definitions improves accuracy, but an isolated explicit state-output field does not significantly improve policy accuracy.
  arXiv  Detail & Related papers  (2026-08-18T01:04:45Z)
• Evaluating and Guarding Citation Faithfulness in Agentic Scientific Synthesis [2.7092559671391]
  Agentic LLM systems such as OpenScholar and PaperQA2 read the scientific literature and return cited answers.<n>Neither audits the reliability of that check itself.<n>We show it is not reliable, and that this matters.<n>We present a gold-anchored evaluation protocol and a deployable guard.
  arXiv  Detail & Related papers  (2026-07-10T02:05:17Z)
• What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility [48.10132234701036]
  We propose a diagnostic evaluation protocol based on three checks -- scale stability, reference validity, and depth utility -- together with per-layer rather than aggregate cosine reporting.<n>Across multiple architectures and methods, our protocol identifies all failures with wide calibration margins.
  arXiv  Detail & Related papers  (2026-06-19T06:04:17Z)
• Send a SCOUT First: Pre-hoc Reasoning for Adaptive Detector Allocation in Prompt-Injection Defense [37.10272384523361]
  We develop a framework for predicting each detector's per-sample reliability and latency from how it behaved on similar past inputs.<n>We build SCOUT-450, a benchmark that captures the structurally complex, agent-facing injections that older prompt-injection sets under-represent.<n>On SCOUT-450, a safety-oriented operating point reduces attack-success rate by 46% and total wall-clock by 40% relative to an always-on GPT-4o judge.
  arXiv  Detail & Related papers  (2026-05-29T04:49:20Z)
• Detecting Is Not Resolving: The Monitoring Control Gap in Retrieval Augmented LLMs [20.59321114618083]
  We show that single-turn diagnostics systematically overestimate RAG safety, that contradiction is uncorrelated with safe resolution, and that no universal prompt fix exists.<n>The gap between what models recognize and what they do must be measured and closed before retrieval-augmented systems can be trusted in high-stakes settings.
  arXiv  Detail & Related papers  (2026-05-26T15:18:43Z)
• Certified Purity for Cognitive Workflow Executors: From Static Analysis to Cryptographic Attestation [0.0]
  A prior three-layer governance architecture proves governance completeness, provenance completeness, and the impossibility of ungoverned effects.<n>This paper closes the gap through four mechanisms.<n>We prove four theorems: structural purity by construction, bypass elimination for all five BEAM bypass classes, certificate integrity, and gate completeness.
  arXiv  Detail & Related papers  (2026-05-01T19:04:37Z)
• Verifier-Bound Communication for LLM Agents: Certified Bounds on Covert Signaling [0.0]
  Colluding language-model agents can hide coordination in messages that remain policy-compliant at the surface level.<n>We present CLBC, a protocol where generation and admission are separated.<n>We show how this protocol yields an upper bound on transcript leakage in terms of latent leakage plus explicit residual channels.
  arXiv  Detail & Related papers  (2026-02-27T23:42:37Z)
• A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness [57.510025257780306]
  We show that existing validation protocols fail to account for substantial distribution shifts inherent to red-teaming.<n>We propose ReliableBench, a benchmark of behaviors that remain more consistently judgeable, and JudgeStressTest, a dataset designed to expose judge failures.
  arXiv  Detail & Related papers  (2026-02-04T15:13:35Z)
• To Throw a Stone with Six Birds: On Agents and Agenthood [0.0]
  Six Birds Theory (SBT) treats macroscopic objects as induced closures rather than primitives.<n>We give a type-correct account of agency within SBT.<n>We operationalize this contract in finite controlled systems using four checkable components.
  arXiv  Detail & Related papers  (2026-02-03T10:46:23Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• Conditional Coverage Diagnostics for Conformal Prediction [47.93989136542648]
  We show that conditional coverage estimation can be a classification problem.<n>We call the resulting family of metrics excess risk of the target coverage (ERT)<n>We release an open-source package for ERT as well as previous conditional coverage metrics.
  arXiv  Detail & Related papers  (2025-12-12T18:47:39Z)
• LEC: Linear Expectation Constraints for False-Discovery Control in Selective Prediction and Routing Systems [95.35293543918762]
  Large language models (LLMs) often generate unreliable answers, while uncertainty methods fail to fully distinguish correct from incorrect predictions.<n>We address this issue through the lens of false discovery rate (FDR) control, ensuring that among all accepted predictions, the proportion of errors does not exceed a target risk level.<n>We propose LEC, which reinterprets selective prediction as a constrained decision problem by enforcing a Linear Expectation Constraint.
  arXiv  Detail & Related papers  (2025-12-01T11:27:09Z)
• Unsupervised Conformal Inference: Bootstrapping and Alignment to Control LLM Uncertainty [49.19257648205146]
  We propose an unsupervised conformal inference framework for generation.<n>Our gates achieve close-to-nominal coverage and provide tighter, more stable thresholds than split UCP.<n>The result is a label-free, API-compatible gate for test-time filtering.
  arXiv  Detail & Related papers  (2025-09-26T23:40:47Z)
• Trusted Uncertainty in Large Language Models: A Unified Framework for Confidence Calibration and Risk-Controlled Refusal [31.458406135473805]
  We present UniCR, a unified framework that turns heterogeneous uncertainty evidence into a calibrated probability of correctness.<n>UniCR learns a lightweight calibration head with temperature scaling and proper scoring.<n>Experiments on short-form QA, code generation with execution tests, and retrieval-augmented long-form QA show consistent improvements in calibration metrics.
  arXiv  Detail & Related papers  (2025-09-01T13:14:58Z)
• Short Paper: Accountable Safety Implies Finality [10.589723476970443]
  Two key desiderata have been studied for Byzantine-fault tolerant (BFT) state-machine replication (SMR) consensus protocols. We show that accountable safety implies finality, thereby unifying earlier results.
  arXiv  Detail & Related papers  (2023-08-31T17:58:38Z)
• Fault-tolerant parity readout on a shuttling-based trapped-ion quantum computer [64.47265213752996]
  We experimentally demonstrate a fault-tolerant weight-4 parity check measurement scheme. We achieve a flag-conditioned parity measurement single-shot fidelity of 93.2(2)%. The scheme is an essential building block in a broad class of stabilizer quantum error correction protocols.
  arXiv  Detail & Related papers  (2021-07-13T20:08:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.