Safe-Night VLA: Seeing the Unseen via Thermal-Perceptive Vision-Language-Action Models for Safety-Critical Manipulation

• URL: http://arxiv.org/abs/2603.05754v1
• Date: Thu, 05 Mar 2026 23:26:44 GMT
• Title: Safe-Night VLA: Seeing the Unseen via Thermal-Perceptive Vision-Language-Action Models for Safety-Critical Manipulation
• Abstract summary: We propose Safe-Night VLA, a multimodal manipulation framework that enables robots to see the unseen.<n> Specifically, Safe-Night VLA integrates long-wave infrared thermal perception into a pre-trained vision-language backbone.<n>We validate our framework through real-world experiments on a Franka manipulator.
• Score: 9.129204825142077
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current Vision-Language-Action (VLA) models rely primarily on RGB perception, preventing them from capturing modalities such as thermal signals that are imperceptible to conventional visual sensors. Moreover, end-to-end generative policies lack explicit safety constraints, making them fragile when encountering obstacles and novel scenarios outside the training distribution. To address these limitations, we propose Safe-Night VLA, a multimodal manipulation framework that enables robots to see the unseen while enforcing rigorous safety constraints for thermal-aware manipulation in unstructured environments. Specifically, Safe-Night VLA integrates long-wave infrared thermal perception into a pre-trained vision-language backbone, enabling semantic reasoning grounded in thermodynamic properties. To ensure safe execution under out-of-distribution conditions, we incorporate a safety filter via control barrier functions, which provide deterministic workspace constraint enforcement during policy execution. We validate our framework through real-world experiments on a Franka manipulator, introducing a novel evaluation paradigm featuring temperature-conditioned manipulation, subsurface target localization, and reflection disambiguation, while maintaining constrained execution at inference time. Results demonstrate that Safe-Night VLA outperforms RGB-only baselines and provide empirical evidence that foundation models can effectively leverage non-visible physical modalities for robust manipulation.

Related papers

• D-SafeMPC: Diffusion-Driven Safe Model Predictive Control with Discrete-Time Control Barrier Functions [35.33769360235326]
  Key limitation on the use of diffusion models in robotic planning is their inability to inherently enforce safety or dynamical constraints.<n>We propose D-SafeMPC, which enhances the interaction between diffusion and control.<n>In simulations on a Franka manipulator, D-SafeMPC improves safety, task success rates, and planning efficiency over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-07-12T17:10:05Z)
• VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoring [60.53395558502203]
  Vision-Language Embodied Safety Agent (VLESA) monitors human activities from egocentric video.<n>VLESA addresses intent-dependent safety where identical actions can be safe or dangerous depending on context.<n>An intent-action prediction agent is proposed to jointly infer goals and predict future actions from video.
  arXiv  Detail & Related papers  (2026-06-02T17:42:17Z)
• SAVeS: Steering Safety Judgments in Vision-Language Models via Semantic Cues [50.928203120795786]
  We study whether multimodal safety behavior in vision-language models can be steered by simple semantic cues.<n>We introduce a semantic steering framework that applies controlled textual, visual, and cognitive interventions without changing the underlying scene content.
  arXiv  Detail & Related papers  (2026-03-19T16:18:00Z)
• BarrierSteer: LLM Safety via Learning Barrier Steering [83.12893815611052]
  BarrierSteer is a novel framework that formalizes safety by embedding learned non-linear safety constraints directly into the model's latent representation space.<n>We show that BarrierSteer substantially reduces adversarial success rates, decreases unsafe generations, and outperforms existing methods.
  arXiv  Detail & Related papers  (2026-02-23T18:19:46Z)
• Contextual Safety Reasoning and Grounding for Open-World Robots [79.98924225712668]
  CORE is a safety framework that enables online contextual reasoning, grounding, and enforcement without prior knowledge of the environment.<n>We provide probabilistic safety guarantees for CORE that account for perceptual uncertainty.<n>We demonstrate through simulation and real-world experiments that CORE enforces contextually appropriate behavior in unseen environments.
  arXiv  Detail & Related papers  (2026-02-23T15:51:23Z)
• Concept-Based Dictionary Learning for Inference-Time Safety in Vision Language Action Models [30.400796921799778]
  Vision Language Action (VLA) models close the perception action loop by translating multimodal instructions into executable behaviors.<n>We introduce a concept-based dictionary learning framework for inference-time safety control.<n> Experiments on Libero-Harm, BadRobot, RoboPair, and IS-Bench show that our approach achieves state-of-the-art defense performance.
  arXiv  Detail & Related papers  (2026-02-02T09:06:43Z)
• Constrained Decoding for Robotics Foundation Models [12.916330118607918]
  We introduce SafeDec, a constrained decoding framework for autoregressive robot foundation models.<n>Task-specific safety rules are expressed as Signal Temporal Logic (STL) formulas and are enforced at inference time with minimal overhead.<n>Our method ensures that generated actions provably satisfy STL specifications under assumed dynamics at runtime without retraining.
  arXiv  Detail & Related papers  (2025-09-01T19:17:40Z)
• Generalizing Safety Beyond Collision-Avoidance via Latent-Space Reachability Analysis [6.267574471145217]
  Hamilton-Jacobi (H) reachability is a rigorous framework that enables robots to simultaneously detect unsafe states and generate actions.<n>We propose La Safety Filters, a latent-space reachability that operates directly on raw observation data.
  arXiv  Detail & Related papers  (2025-02-02T22:00:20Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)
• In-Distribution Barrier Functions: Self-Supervised Policy Filters that Avoid Out-of-Distribution States [84.24300005271185]
  We propose a control filter that wraps any reference policy and effectively encourages the system to stay in-distribution with respect to offline-collected safe demonstrations. Our method is effective for two different visuomotor control tasks in simulation environments, including both top-down and egocentric view settings.
  arXiv  Detail & Related papers  (2023-01-27T22:28:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.