Forecasting Trajectory-Level Safety Risks in Black-Box Multi-Turn Interactions

• URL: http://arxiv.org/abs/2607.26820v1
• Date: Wed, 29 Jul 2026 12:14:23 GMT
• Title: Forecasting Trajectory-Level Safety Risks in Black-Box Multi-Turn Interactions
• Abstract summary: We propose Recast, a safety risk forecasting framework for large language models (LLMs)<n>Recast predicts 88.3% of future safety failures with an average lead time of 2.41 turns, while maintaining a false alarm rate of 12.3%.
• Score: 13.366980144986707
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As large language models (LLMs) evolve from standalone assistants into autonomous agents, ensuring their safety requires shifting beyond pointwise risk assessment to understand how risks emerge and unfold over long-horizon trajectories. In multi-turn interactions, malicious intent can be decomposed across seemingly harmless turns and gradually reconstructed through interaction trajectories, eventually resulting in safety failures. Existing safeguards remain largely reactive, detecting manifested violations while lacking the ability to predict latent risk evolution and enable preemptive prevention. To address this limitation, we propose Recast, a safety risk forecasting framework that advances LLM safeguarding beyond turn-level violation detection to trajectory-level risk prediction. Recast first retrieves risk-relevant evidence from both short-term dialogue progression and long-term historical context via a dual-scale trajectory view. It then models compositional risk evolution by capturing the current risk configuration and its temporal dynamics. Finally, a causal temporal encoder learns latent risk evolution patterns and predicts the distribution of future risk emergence turns. Extensive experiments across 7 risk categories show that Recast predicts 88.3% of future safety failures with an average lead time of 2.41 turns, while maintaining a false alarm rate of 12.3%, showcasing the effectiveness of trajectory-level forecasting in identifying emerging risks before safety violations occur.

Related papers

• TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modeling [18.58206519209223]
  LLM agents increasingly operate through multi-turn tool use and environment interaction.<n>Post-hoc diagnosis frequently misses the chance to flag risks while they are unfolding.<n>We propose TRACES, a representation-based proactive auditor.
  arXiv  Detail & Related papers  (2026-05-26T21:11:02Z)
• Action-Conditioned Risk Gating for Safety-Critical Control under Partial Observability [79.08785366532287]
  We propose a lightweight risk-gated reinforcement learning approximation for risk-sensitive control under partial observability.<n>We evaluate the approach in two safety-critical partially observable domains: automated glucose regulation and safety-constrained navigation.
  arXiv  Detail & Related papers  (2026-05-14T01:23:09Z)
• RiskProp: Collision-Anchored Self-Supervised Risk Propagation for Early Accident Anticipation [16.390547433087185]
  Accident anticipation aims to predict impending collisions from dashcam videos and trigger early alerts.<n>Existing methods rely on binary supervision with manually annotated "anomaly onset" frames.<n>We propose RiskProp, a novel collision-anchored self-supervised risk propagation paradigm for early accident anticipation.
  arXiv  Detail & Related papers  (2026-03-28T07:04:16Z)
• Uncertainty-Aware Vision-based Risk Object Identification via Conformal Risk Tube Prediction [7.840136094141999]
  Existing approaches make decisions and ignore uncertainty, which could lead to safety-critical failures.<n>Despite these challenges, current methods lack principled framework to model risk uncertainty jointly across space and time.<n>We propose Conformal Risk Prediction unified formulation that captures principled risk uncertainty, provides coverage guarantees for true risks, and produces calibrated risk scores with uncertainty estimates.
  arXiv  Detail & Related papers  (2026-03-25T04:13:42Z)
• The Missing Half: Unveiling Training-time Implicit Safety Risks Beyond Deployment [148.80266237240713]
  implicit training-time safety risks are driven by a model's internal incentives and contextual background information.<n>We present the first systematic study of this problem, introducing a taxonomy with five risk levels, ten fine-grained risk categories, and three incentive types.<n>Our results identify an overlooked yet urgent safety challenge in training.
  arXiv  Detail & Related papers  (2026-02-04T04:23:58Z)
• SafePred: A Predictive Guardrail for Computer-Using Agents via World Models [12.569157125705052]
  We present SafePred, a predictive guardrail framework for Computer-using Agents (CUAs) in complex real-world environments.<n>Based on this approach, we present SafePred that establishes a risk-to-decision loop to ensure safe agent behavior.<n>Extensive experiments show that SafePred significantly reduces high-risk behaviors, achieving over 97.6% safety performance and improving task utility by up to 21.4% compared with reactive baselines.
  arXiv  Detail & Related papers  (2026-02-02T07:04:06Z)
• Constrained Language Model Policy Optimization via Risk-aware Stepwise Alignment [49.2305683068875]
  We propose Risk-aware Stepwise Alignment (RSA), a novel alignment method that incorporates risk awareness into the policy optimization process.<n> RSA mitigates risks induced by excessive model shift away from a reference policy, and it explicitly suppresses low-probability yet high-impact harmful behaviors.<n> Experimental results demonstrate that our method achieves high levels of helpfulness while ensuring strong safety.
  arXiv  Detail & Related papers  (2025-12-30T14:38:02Z)
• When Models Outthink Their Safety: Mitigating Self-Jailbreak in Large Reasoning Models with Chain-of-Guardrails [74.63933201261595]
  Large Reasoning Models (LRMs) demonstrate remarkable capabilities on complex reasoning tasks.<n>LRMs remain vulnerable to severe safety risks, including harmful content generation and jailbreak attacks.<n>We propose the Chain-of-Guardrail (CoG), a training framework that recomposes or backtracks unsafe reasoning steps.
  arXiv  Detail & Related papers  (2025-10-24T09:32:25Z)
• Building a Foundational Guardrail for General Agentic Systems via Synthetic Data [76.18834864749606]
  LLM agents can plan multi-step tasks, intervening at the planning stage-before any action is executed-is often the safest way to prevent harm.<n>Existing guardrails mostly operate post-execution, which is difficult to scale and leaves little room for controllable supervision at the plan level.<n>We introduce AuraGen, a controllable engine that synthesizes benign trajectories, injects category-labeled risks with difficulty, and filters outputs via an automated reward model.
  arXiv  Detail & Related papers  (2025-10-10T18:42:32Z)
• STRAP: Spatial-Temporal Risk-Attentive Vehicle Trajectory Prediction for Autonomous Driving [0.968535561940627]
  We propose a novel spatial-temporal risk-attentive trajectory prediction framework that incorporates a risk potential field.<n>The proposed framework provides interpretable, risk-aware predictions, contributing to more robust decision-making for autonomous driving systems.
  arXiv  Detail & Related papers  (2025-07-11T13:05:35Z)
• Efficient Risk-Averse Reinforcement Learning [79.61412643761034]
  In risk-averse reinforcement learning (RL), the goal is to optimize some risk measure of the returns. We prove that under certain conditions this inevitably leads to a local-optimum barrier, and propose a soft risk mechanism to bypass it. We demonstrate improved risk aversion in maze navigation, autonomous driving, and resource allocation benchmarks.
  arXiv  Detail & Related papers  (2022-05-10T19:40:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.