GroundControl: Anticipating Navigation Failures in Vision-Language Agents via Trajectory-Consistent Uncertainty Estimates

• URL: http://arxiv.org/abs/2606.20479v1
• Date: Thu, 18 Jun 2026 16:56:20 GMT
• Title: GroundControl: Anticipating Navigation Failures in Vision-Language Agents via Trajectory-Consistent Uncertainty Estimates
• Abstract summary: Vision-language navigation agents achieve competitive average success on benchmark tasks, yet failures often arise through predictable trajectory-level breakdowns.<n>Reliable deployment requires uncertainty signals that anticipate emerging failure dynamics during execution rather than reflect only instantaneous action entropy.<n>We introduce emphGroundControl, a trajectory-consistent uncertainty estimator defined as statistical deviation from nominal goal-directed distance-to-goal dynamics aggregated over an episode.
• Score: 3.7267095596919053
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language navigation agents achieve competitive average success on benchmark tasks, yet failures often arise through predictable trajectory-level breakdowns such as oscillation, stagnation, or inefficient detours. Reliable deployment, therefore, requires uncertainty signals that anticipate emerging failure dynamics during execution rather than reflect only instantaneous action entropy. We introduce \emph{GroundControl}, a trajectory-consistent uncertainty estimator defined as statistical deviation from nominal goal-directed distance-to-goal dynamics aggregated over an episode. GroundControl models distance evolution using a constant-velocity Kalman filter and combines normalized innovation statistics with complementary trajectory features capturing progress, monotonicity, path efficiency, and oscillatory behavior. The resulting uncertainty score reflects geometric and temporal inconsistency in navigation behavior rather than local prediction dispersion. To evaluate uncertainty quality independently of task success, we formalize \emph{Selective Risk--Coverage Navigation (SRCN)}, a protocol that measures how effectively an uncertainty score ranks episodes by failure or inefficiency using risk--coverage curves and AURC / E-AURC summaries. Across five EB-Navigation splits ($N=300$ episodes), trajectory-consistent uncertainty achieves near-oracle ordering under success-based selective risk, with weighted-average $\mathrm{E\text{-}AURC}_{\mathrm{SR}}=0.0024$ for the GPT-4o model, substantially outperforming entropy-, conformal-, and heuristic baselines. Under SPL-based selective evaluation, GroundControl consistently achieves the lowest AURC and E-AURC across models and navigation splits. These results show that modeling deviation from goal-directed dynamics provides an interpretable and robust signal for anticipating navigation failures in vision-language agents.

Related papers

• IADD-TR: Intervention-Aware Dynamics Decoupling with Targeted Regularization for Model-Based Reinforcement Learning [57.16513298507272]
  We propose IADD-TR, a unified framework combining Intervention-Aware Dynamics Decoupling (IADD) and Targeted Regularization (TR)<n>IADD factorizes transitions into an action-intervention stage and an action-free natural evolution stage, using a zero-action anchor to resolve the non-uniqueness of this two-stage factorization for robust generalization.<n>For policy learning, we derive TR from the efficient influence function of a replay-state policy-gradient functional.
  arXiv  Detail & Related papers  (2026-08-11T08:20:02Z)
• Nonlinear Model Predictive Control via Sequential Convex Programming for Drone-to-Drone Docking [0.0]
  This work formulates the docking task as a finite-horizon optimal control problem based on a reduced-order nonlinear model augmented with disturbance states.<n>The resulting problem is solved using sequential convex programming within a receding-horizon framework to generate dynamically feasible docking trajectories.
  arXiv  Detail & Related papers  (2026-08-11T06:27:45Z)
• Shift & Drift: A Zero-Shot Benchmark for Generalizable and Robust Autonomous Driving Motion Planning [60.888463449435484]
  We present Shift & Drift, a novel dual-track benchmark designed to rigorously stress-test motion planners across two critical axes of distribution shift.<n>We show that imitation learning methods achieve high scores in ID benchmarks, but suffer from persistent drift when subjected to temporally correlated actuation noise.<n>Our findings reveal an empirical trade-off between imitation fidelity and closed-loop resilience, providing the community with a rigorous benchmark to evaluate progress toward reliable deployment.
  arXiv  Detail & Related papers  (2026-07-08T18:23:48Z)
• Trust, Geometry, and Rules: A Credibility-Aware Reinforcement Learning Framework for Safe USV Navigation under Uncertainty [32.34854002457735]
  We propose a framework integrating credibility-aware learning, geometric safety shielding, and continuous rule-aware embedding.<n>We show improved training robustness against perceptual inconsistency and superior collision avoidance and COLREGs compliance over baselines.
  arXiv  Detail & Related papers  (2026-05-26T12:59:43Z)
• Mitigating Error Amplification in Fast Adversarial Training [58.74042726356826]
  Fast Adversarial Training (FAT) has proven effective in enhancing model robustness by encouraging networks to learn perturbation-invariant representations.<n>FAT often suffers from catastrophic overfitting (CO), where the model overfits to the training attack and fails to generalize to unseen ones.<n>We propose a Distribution-aware Dynamic Guidance (DDG) strategy that dynamically adjusts both the perturbation budget and supervision signal.
  arXiv  Detail & Related papers  (2026-04-27T11:23:18Z)
• AnchorRefine: Synergy-Manipulation Based on Trajectory Anchor and Residual Refinement for Vision-Language-Action Models [60.04879435087352]
  Vision-language-action (VLA) policies generate actions within a single unified space.<n>We propose AnchorRefine, a hierarchical framework that factorizes VLA action modeling into trajectory anchor and residual refinement.<n> Experiments on LIBERO, CALVIN, and real-robot tasks demonstrate that AnchorRefine consistently improves both regression-based and diffusion-based VLA backbones.
  arXiv  Detail & Related papers  (2026-04-20T04:25:24Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• Forecasting Fails: Unveiling Evasion Attacks in Weather Prediction Models [60.728124907335]
  This work introduces Weather Adaptive Adversarial Perturbation Optimization (WAAPO), a novel framework for generating targeted adversarial perturbations.<n>WAAPO achieves this by incorporating constraints for channel sparsity, spatial localization, and smoothness, ensuring that perturbations remain physically realistic and imperceptible.<n>Our experiments highlight critical vulnerabilities in AI-driven forecasting models, where small perturbations to initial conditions can result in significant deviations.
  arXiv  Detail & Related papers  (2025-12-09T17:20:56Z)
• ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Driving [64.42138266293202]
  ResAD is a Normalized Residual Trajectory Modeling framework.<n>It reframes the learning task to predict the residual deviation from an inertial reference.<n>On the NAVSIM benchmark, ResAD achieves a state-of-the-art PDMS of 88.6 using a vanilla diffusion policy.
  arXiv  Detail & Related papers  (2025-10-09T17:59:36Z)
• Active Test-time Vision-Language Navigation [60.69722522420299]
  ATENA is a test-time active learning framework that enables a practical human-robot interaction via episodic feedback on uncertain navigation outcomes.<n>In particular, ATENA learns to increase certainty in successful episodes and decrease it in failed ones, improving uncertainty calibration.<n>In addition, we propose a self-active learning strategy that enables an agent to evaluate its navigation outcomes based on confident predictions.
  arXiv  Detail & Related papers  (2025-06-07T02:24:44Z)
• Interpretable Stochastic Model Predictive Control using Distributional Reinforced Estimation for Quadrotor Tracking Systems [0.8411385346896411]
  We present a novel trajectory tracker for autonomous quadrotor navigation in dynamic and complex environments. The proposed framework integrates a distributional Reinforcement Learning estimator for unknown aerodynamic effects into a Model Predictive Controller. We demonstrate our system to improve the cumulative tracking errors by at least 66% with unknown and diverse aerodynamic forces.
  arXiv  Detail & Related papers  (2022-05-14T23:27:38Z)
• Guaranteed Trajectory Tracking under Learned Dynamics with Contraction Metrics and Disturbance Estimation [5.147919654191323]
  This paper presents an approach to trajectory-centric learning control based on contraction metrics and disturbance estimation. The proposed framework is validated on a planar quadrotor example.
  arXiv  Detail & Related papers  (2021-12-15T15:57:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.