Event-Adaptive Motion Planning with Distilled Vision-Language Model in Safety-Critical Situations

• URL: http://arxiv.org/abs/2606.25629v1
• Date: Wed, 24 Jun 2026 09:35:26 GMT
• Title: Event-Adaptive Motion Planning with Distilled Vision-Language Model in Safety-Critical Situations
• Abstract summary: Large vision-language models (VLMs) offer remarkable capabilities in commonsense reasoning.<n>We propose event-adaptive motion planning (EAMP), an efficient framework for VLM-based robot navigation.<n>EAMP effectively aligns high-level reasoning with low-level control, significantly improving dynamic safety margins.
• Score: 25.961619880053235
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robot navigation in safety-critical scenarios faces significant challenges from unforeseen semantic events, where collisions arise primarily from the unpredictable behaviors of dynamic agents rather than unseen objects. While large vision-language models (VLMs) offer remarkable capabilities in commonsense reasoning, frequently invoking them within the continuous control loop introduces severe computational latency, fundamentally destabilizing physical execution. To address these challenges, we propose event-adaptive motion planning (EAMP), an efficient framework for VLM-based robot navigation. Specifically, a prompt-configurable semantic event trigger (PC-SET) selectively activates semantic intervention by continuously monitoring short temporal clips for behavioral anomalies. Upon triggering, an event-triggered distilled SemNav-VLM, fine-tuned via physically verified semantic distillation, maps detected anomalies into discrete strategy-level decisions. Subsequently, a semantic model predictive control (SMPC) module translates these strategies into dynamic reconfigurations of optimization objectives and geometric references. Extensive experiments in safety-critical logistics scenarios demonstrate that EAMP effectively aligns high-level reasoning with low-level control, significantly improving dynamic safety margins over existing baselines while preserving real-time efficiency.

Related papers

• Learning-Based Dynamic Obstacle Avoidance for a UAV Using Only Three Range Sensors [1.2246649738388389]
  We present a learning-based approach to kinodynamic online motion planning for an Unmanned Aerial Vehicle (UAV) operating at a fixed altitude in unknown dynamic environments.<n>The proposed framework integrates a behavior grid map representation with Deep Reinforcement Learning (DRL) for stable policy learning in continuous control.<n>The key idea is the co-design of a state representation and control policy that enables reliable navigation using only three low-cost directional range sensors.
  arXiv  Detail & Related papers  (2026-09-13T10:53:43Z)
• GoStop: Reinforcement Learning for Adaptive Temporal Aggregation in Event-Based Feature Tracking [71.9852231453445]
  Event cameras enable low-latency and motion-robust perception, making them well-suited for feature tracking under fast and non-linear motion.<n>Existing event-based feature tracking methods rely on fixed rules based on hand-tuning for event accumulation.<n>We introduce reinforcement learning framework to adaptively control the accumulation process for online event-based feature tracking.
  arXiv  Detail & Related papers  (2026-07-17T07:18:07Z)
• Proposal-Conditioned Latent Diffusion for Closed-Loop Traffic Scenario Generation [1.6711881128691604]
  We present a diffusion-based scenario generation framework conditioned on instance-centric scene context and multimodal proposal priors.<n>Experiments on the Open Motion dataset demonstrate a favorable balance among realism, safety, and controllability across diverse interactive scenarios.
  arXiv  Detail & Related papers  (2026-06-25T15:01:31Z)
• GuardAD: Safeguarding Autonomous Driving MLLMs via Markovian Safety Logic [51.85782253444399]
  Multimodal large language models (MLLMs) are increasingly integrated into autonomous driving systems.<n>They remain vulnerable to diverse safety threats, particularly in accident-prone scenarios.<n>We propose GuardAD, a model-agnostic safeguard that formulates AD safety as an evolving Markovian logical state.
  arXiv  Detail & Related papers  (2026-05-11T11:28:03Z)
• F2F-AP: Flow-to-Future Asynchronous Policy for Real-time Dynamic Manipulation [62.06267255986041]
  Asynchronous inference has emerged as a prevalent paradigm in robotic manipulation.<n>In this paper, we propose a novel framework that leverages predicted object flow to synthesize future observations.<n>Our approach significantly enhances responsiveness and success rates in complex dynamic manipulation tasks.
  arXiv  Detail & Related papers  (2026-04-02T17:57:15Z)
• Reshaping Action Error Distributions for Reliable Vision-Language-Action Models [69.38615670891038]
  In robotic manipulation, vision-language-action (VLA) models have emerged as a promising paradigm for learning generalizable and scalable robot policies.<n>We focus on continuous-action VLA models and move beyond conventional MSE-based regression by reshaping action error distributions during training.<n>We evaluate our approaches across standard, few-shot, and noisy settings on multiple representative VLA architectures.
  arXiv  Detail & Related papers  (2026-02-04T05:37:09Z)
• SPOT: Spatio-Temporal Obstacle-free Trajectory Planning for UAVs in an Unknown Dynamic Environment [1.9181913148426697]
  We develop a 4-dimensional planner, integrated with vision-based Safe Flight Corridor (SFC) generation.<n>We validate our method extensively in both simulation and real-world hardware experiments and benchmark it against state-of-the-art approaches.
  arXiv  Detail & Related papers  (2026-02-01T12:24:12Z)
• \textsc{NaVIDA}: Vision-Language Navigation with Inverse Dynamics Augmentation [50.027425808733994]
  textscNaVIDA is a unified VLN framework that couples policy learning with action-grounded visual dynamics and adaptive execution.<n>textscNaVIDA augments training with chunk-based inverse-dynamics supervision to learn causal relationship between visual changes and corresponding actions.<n>Experiments show that textscNaVIDA achieves superior navigation performance compared to state-of-the-art methods with fewer parameters.
  arXiv  Detail & Related papers  (2026-01-26T06:16:17Z)
• EASE: Embodied Active Event Perception via Self-Supervised Energy Minimization [6.249768559720122]
  Active event perception is essential for embodied intelligence in tasks such as human-AI collaboration, assistive robotics, and autonomous navigation.<n>We propose EASE, a self-supervised framework that unifies representation learning and embodied control through free energy.
  arXiv  Detail & Related papers  (2025-06-20T23:45:51Z)
• Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning [18.28480383898768]
  FORTRESS is a joint reasoning and planning framework for robot safety.<n>It generates semantically safe fallback strategies to prevent safety-critical, OOD failures.
  arXiv  Detail & Related papers  (2025-05-15T17:55:28Z)
• DynaVINS++: Robust Visual-Inertial State Estimator in Dynamic Environments by Adaptive Truncated Least Squares and Stable State Recovery [11.37707868611451]
  We propose a robust VINS framework called mboxtextitDynaVINS++. Our approach shows promising performance in dynamic environments, including scenes with abruptly dynamic objects.
  arXiv  Detail & Related papers  (2024-10-20T12:13:45Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.