Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs

• URL: http://arxiv.org/abs/2604.07655v1
• Date: Wed, 08 Apr 2026 23:47:29 GMT
• Title: Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs
• Abstract summary: Hard-gated safety checkers often over-refuse and misalign with a vendor's model spec.<n>This work introduces Guardian-as-an-Advisor (GaaA), a soft-gating pipeline where a guardian predicts a binary risk label and prepends this advice to the original query for re-inference.<n>Overall, GaaA steers models to comply with the model spec, maintaining safety while reducing over-refusal.
• Score: 70.81495077853673
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hard-gated safety checkers often over-refuse and misalign with a vendor's model spec; prevailing taxonomies also neglect robustness and honesty, yielding safer-on-paper yet less useful systems. This work introduces Guardian-as-an-Advisor (GaaA), a soft-gating pipeline where a guardian predicts a binary risk label plus a concise explanation and prepends this advice to the original query for re-inference, keeping the base model operating under its original spec. To support training and evaluation, GuardSet is constructed, a 208k+ multi-domain dataset unifying harmful and harmless cases with targeted robustness and honesty slices. GuardAdvisor is trained via SFT followed by RL to enforce label-explanation consistency. GuardAdvisor attains competitive detection accuracy while enabling the advisory workflow; when used to augment inputs, responses improve over unaugmented prompts. A latency study shows advisor inference uses below 5% of base-model compute and adds only 2-10% end-to-end overhead under realistic harmful-input rates. Overall, GaaA steers models to comply with the model spec, maintaining safety while reducing over-refusal.

Related papers

• Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models [31.940554924036807]
  Large Reasoning Models (LRMs) remain highly vulnerable to sophisticated jailbreaks and direct harmful queries.<n>We employSupervised Fine-Tuning (SFT) to explicitly induce safe tags to trigger safety analysis and guidance.<n>We also apply Direct Preference Optimization (DPO) to further enhance the correctness and stability of the safety analysis and guidance.
  arXiv  Detail & Related papers  (2026-06-15T14:51:34Z)
• Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Models [55.788110316999166]
  We compare reasoning models produced via supervised fine-tuning, RL-based post-training, and distillation against instruction-tuned baselines.<n>We observe that reasoning models often improve on reasoning benchmarks but exhibit alignment regressions.<n>These regressions are consistent with behavioral drift from the instruction-tuned baseline, measured by KL divergence.
  arXiv  Detail & Related papers  (2026-06-09T16:14:27Z)
• DDOR: Delta Debugging for Explainable Overrefusal Testing and Repair [17.573294728864443]
  DDOR (Delta for OverRefusal) is a fully automated and explainable framework for overrefusal testing and repair.<n>It applies delta debug to localize minimal refusal-triggering fragments (mRTFs) that provide phrase-level, explainable evidence for why a refusal occurs.<n>It produces scalable and model-specific overrefusal test suites (approximately 1K cases per model)
  arXiv  Detail & Related papers  (2026-06-02T13:07:12Z)
• THINKSAFE: Self-Generated Safety Alignment for Reasoning Models [60.10077024249373]
  We propose ThinkSafe, a framework that restores safety alignment without external teachers.<n>Our key insight is that while compliance suppresses safety mechanisms, models often retain latent knowledge to identify harm.<n> Experiments on DeepSeek-R1-Distill and Qwen3 show ThinkSafe significantly improves safety while preserving reasoning proficiency.
  arXiv  Detail & Related papers  (2026-01-30T16:31:02Z)
• ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack [52.17935054046577]
  We present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks.<n>ReasAlign incorporates structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks.
  arXiv  Detail & Related papers  (2026-01-15T08:23:38Z)
• AprielGuard [2.3704817495377526]
  Existing tools treat safety risks as separate problems, limiting robustness and generalizability.<n>We introduce AprielGuard, an 8B parameter safeguard model that unify these dimensions within a single taxonomy and learning framework.<n> AprielGuard achieves strong performance in detecting harmful content and adversarial manipulations.
  arXiv  Detail & Related papers  (2025-12-23T12:01:32Z)
• Bag of Tricks for Subverting Reasoning-based Safety Guardrails [62.139297207938036]
  We present a bag of jailbreak methods that subvert the reasoning-based guardrails.<n>Our attacks span white-, gray-, and black-box settings and range from effortless template manipulations to fully automated optimization.
  arXiv  Detail & Related papers  (2025-10-13T16:16:44Z)
• RAG Makes Guardrails Unsafe? Investigating Robustness of Guardrails under RAG-style Contexts [39.58550043591753]
  External LLM-based guardrail models have emerged as a popular solution to screen unsafe inputs and outputs.<n>We investigated how robust LLM-based guardrails are against additional information embedded in the context.
  arXiv  Detail & Related papers  (2025-10-06T19:20:43Z)
• ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrails [33.96886111900147]
  ThinkGuard is a critique-augmented guardrail model that distills knowledge from high-capacity language models.<n>It achieves the highest average F1 and AUPRC, outperforming all baselines.<n>It surpasses label-only fine-tuned models, confirming that structured critiques enhance both classification precision and nuanced safety reasoning.
  arXiv  Detail & Related papers  (2025-02-19T06:09:58Z)
• Jailbreaking as a Reward Misspecification Problem [80.52431374743998]
  We propose a novel perspective that attributes this vulnerability to reward misspecification during the alignment process.<n>We introduce a metric ReGap to quantify the extent of reward misspecification and demonstrate its effectiveness.<n>We present ReMiss, a system for automated red teaming that generates adversarial prompts in a reward-misspecified space.
  arXiv  Detail & Related papers  (2024-06-20T15:12:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.