COMPASS: Cognitive MCTS-Guided Process Alignment for Safe Search Agents

• URL: http://arxiv.org/abs/2605.30838v1
• Date: Fri, 29 May 2026 04:51:06 GMT
• Title: COMPASS: Cognitive MCTS-Guided Process Alignment for Safe Search Agents
• Abstract summary: LLM-powered search agents enable multi-step reasoning and tool use.<n>Existing alignment methods struggle to capture sparse safety signals and fail to supervise diverse violations across multi-step interactions.<n>We propose a Cognitive MCTS-Guided Process Alignment framework to achieve robust safety alignment throughout the agent workflow.
• Score: 22.338984431904816
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLM-powered search agents enable multi-step reasoning and tool use. However, these capabilities introduce retrieval-induced safety degradation, as harmful intents may decompose into seemingly innocuous sub-queries that lead to unsafe outcomes. Existing alignment methods struggle to capture sparse safety signals and fail to supervise diverse violations across multi-step interactions. We propose COMPASS, a Cognitive MCTS-Guided Process Alignment framework designed to achieve robust safety alignment throughout the agent workflow while preserving general utility. COMPASS integrates cognitive tree exploration (CTE) to efficiently synthesize stealthy attack trajectories, and introspective step-wise alignment (ISA) to isolate risky intermediate actions for fine-grained process supervision. Empirical results show that COMPASS achieves a favorable safety-utility trade-off while requiring substantially less training data.

Related papers

• AgentLens: Interpretable Safety Steering via Mechanistic Subspaces for Multi-Turn Coding Agent [53.82005364479556]
  Coding agents based on large language models (LLMs) demonstrate remarkable autonomous capabilities.<n>Existing safety mechanisms mainly rely on external guardrails.<n>We propose AgentLens, a white-box defense framework that performs runtime safety detection and representation-level mitigation.
  arXiv  Detail & Related papers  (2026-06-21T21:13:55Z)
• RUBAS: Rubric-Based Reinforcement Learning for Agent Safety [93.80584563695646]
  RUBAS is a rubric-based reinforcement learning framework for agent safety.<n>It decomposes agent behavior into four dimensions: tool-use safety, argument safety, response safety, and helpfulness.<n>Experiments show that RUBAS improves safety over standard alignment baselines, reduces tool-grounded hallucinations, and maintains competitive utility.
  arXiv  Detail & Related papers  (2026-06-02T09:02:14Z)
• TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modeling [18.58206519209223]
  LLM agents increasingly operate through multi-turn tool use and environment interaction.<n>Post-hoc diagnosis frequently misses the chance to flag risks while they are unfolding.<n>We propose TRACES, a representation-based proactive auditor.
  arXiv  Detail & Related papers  (2026-05-26T21:11:02Z)
• BarrierSteer: LLM Safety via Learning Barrier Steering [83.12893815611052]
  BarrierSteer is a novel framework that formalizes safety by embedding learned non-linear safety constraints directly into the model's latent representation space.<n>We show that BarrierSteer substantially reduces adversarial success rates, decreases unsafe generations, and outperforms existing methods.
  arXiv  Detail & Related papers  (2026-02-23T18:19:46Z)
• DRAFT: Task Decoupled Latent Reasoning for Agent Safety [59.46137757545185]
  We propose DRAFT (Task Decoupled Latent Reasoning for Agent Safety), a latent reasoning framework that decouples safety judgment into two trainable stages.<n>An Extractor distills the full trajectory into a compact continuous latent draft, and a Reasoner that jointly attends to the draft and the original trajectory to predict safety.<n>DRAFT consistently outperforms strong baselines, improving accuracy from 63.27% (LoRA) to 91.18% averaged over benchmarks.
  arXiv  Detail & Related papers  (2026-02-11T07:45:14Z)
• How Brittle is Agent Safety? Rethinking Agent Risk under Intent Concealment and Task Complexity [55.441602598245744]
  Current safety evaluations for LLM-driven agents primarily focus on atomic harms, failing to address sophisticated threats where malicious intent is concealed or diluted within complex tasks.<n>We address this gap with a two-dimensional analysis of agent safety brittleness under the pressures of intent concealment and task complexity.<n>Our findings reveal two critical phenomena: safety alignment degrades sharply and predictably as intent becomes obscured, and a "Complexity Paradox" emerges, where agents seem safer on harder tasks only due to capability limitations.
  arXiv  Detail & Related papers  (2025-11-11T17:27:27Z)
• LatentGuard: Controllable Latent Steering for Robust Refusal of Attacks and Reliable Response Generation [4.29885665563186]
  LATENTGUARD is a framework that combines behavioral alignment with supervised latent space control for interpretable and precise safety steering.<n>Our results show significant improvements in both safety controllability and response interpretability without compromising utility.
  arXiv  Detail & Related papers  (2025-09-24T07:31:54Z)
• Efficient Switchable Safety Control in LLMs via Magic-Token-Guided Co-Training [1.5349686675266894]
  Current methods for content safety in Large Language Models (LLMs) rely on multi-stage training pipelines.<n>We propose a unified co-training framework that efficiently integrates multiple safety behaviors.<n>We show that our method matches the safety alignment quality of SFT+DPO, with our 8B model notably surpassing DeepSeek-R1 (671B) in safety performance.
  arXiv  Detail & Related papers  (2025-08-12T02:39:33Z)
• SAFER: Advancing Safety Alignment via Efficient Ex-Ante Reasoning [51.78514648677898]
  We propose SAFER, a framework for Safety Alignment via eFficient Ex-Ante Reasoning.<n>Our approach instantiates structured Ex-Ante reasoning through initial assessment, rule verification, and path calibration.<n> Experiments on multiple open-source LLMs demonstrate that SAFER significantly enhances safety performance while maintaining helpfulness and response efficiency.
  arXiv  Detail & Related papers  (2025-04-03T16:07:38Z)
• SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering [56.92068213969036]
  Safety alignment is indispensable for Large Language Models (LLMs) to defend threats from malicious instructions.<n>Recent researches reveal safety-aligned LLMs prone to reject benign queries due to the exaggerated safety issue.<n>We propose a Safety-Conscious Activation Steering (SCANS) method to mitigate the exaggerated safety concerns.
  arXiv  Detail & Related papers  (2024-08-21T10:01:34Z)
• Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesis [63.532413807686524]
  This paper addresses the problem of maintaining safety during training in Reinforcement Learning (RL) We propose a new architecture that handles the trade-off between efficient progress and safety during exploration.
  arXiv  Detail & Related papers  (2023-12-18T16:09:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.