sensVLA: Spatially-Grounded Vision-Language-Action Model for Autonomous Wheel Loader

• URL: http://arxiv.org/abs/2609.17021v1
• Date: Tue, 15 Sep 2026 11:29:50 GMT
• Title: sensVLA: Spatially-Grounded Vision-Language-Action Model for Autonomous Wheel Loader
• Abstract summary: We present sensVLA, a Vision-Language-Action architecture for autonomous wheel-loader control.<n>It combines a Qwen3-2B Vision-Language Model (VLM) with a fully trainable transformer action expert trained by flow-matching velocity regression.<n>sensVLA routes Bird's-Eye-View (BEV) features, extracted from fused front and rear lidar, directly to the action expert through a dedicated cross-attention pathway.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Autonomous wheel-loader control requires joint reasoning over task semantics, egocentric vision, proprioception, and 3D scene geometry. We present sensVLA, a Vision-Language-Action (VLA) architecture that combines a Qwen3-2B Vision-Language Model (VLM) with a fully trainable transformer action expert trained by flow-matching velocity regression. sensVLA routes Bird's-Eye-View (BEV) features, extracted from fused front and rear lidar, directly to the action expert through a dedicated cross-attention pathway, while the VLM consumes front and rear RGB views to provide task-conditioned semantic context. This design decouples spatial grounding from linguistic reasoning while preserving interaction between both streams at decision time. The expert predicts six action dimensions: longitudinal velocity, steering, body-frame displacement, arm rate, and bucket rate. On a real-world dataset from a wheel loader, sensVLA reaches aggregate per-step parity with a strong camera-only baseline and reduces longitudinal velocity RMSE by 28% and displacement error by 9% on loading centric scenarios. It also degrades 29% less when the camera stream is corrupted or removed, evidencing that explicit spatial grounding improves accuracy and fault-tolerance for heavy equipment autonomy.

Related papers

• ReViV: Reconstructing the Viewer and the View in 4D from Monocular Egocentric Video [51.951128822418774]
  ReViV is a framework for holistic egocentric 4D reconstruction.<n>Powered by a Masked Generative Egocentric Transformer, ReViV operates within a single feed-forward architecture.
  arXiv  Detail & Related papers  (2026-07-20T10:22:53Z)
• MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving [54.57163800903507]
  We present MindVLA-U1, the first unified streaming VLA architecture for autonomous driving.<n>A unified VLM backbone produces AR language tokens and flow-matching continuous action trajectories in a single forward pass over one shared representation.<n>On the long-tail WOD-E2E benchmark, MindVLA-U1 surpasses experienced human drivers for the first time.
  arXiv  Detail & Related papers  (2026-05-12T18:09:42Z)
• UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving [48.461267171124945]
  We propose UniDriveVLA, a Unified Driving Vision-Language-Action model for autonomous driving.<n>It comprises three experts for driving understanding, scene perception, and action planning, which are coordinated through masked joint attention.<n>It achieves state-of-the-art performance in open-loop evaluation on nuScenes and closed-loop evaluation on Bench2Drive.
  arXiv  Detail & Related papers  (2026-04-02T15:48:45Z)
• AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Control [20.1849703990752]
  Vision-Language Navigation (VLN) for Unmanned Aerial Vehicles (UAVs) demands complex visual interpretation and continuous control in 3D environments.<n>We propose AerialVLA, a minimalist end-to-end Vision-Language-Action framework mapping raw visual observations and fuzzy linguistic instructions directly to continuous physical control signals.
  arXiv  Detail & Related papers  (2026-03-15T13:02:13Z)
• AnchorVLA4D: an Anchor-Based Spatial-Temporal Vision-Language-Action Model for Robotic Manipulation [11.15252844796209]
  Vision-Language-Action (VLA) systems suffer from limited spatial perception and the absence of memory throughout manipulation.<n>We propose AnchorVLA4D, a simple spatial-temporal VLA that augments the visual input with an anchor image to preserve the initial scene context.
  arXiv  Detail & Related papers  (2026-03-13T07:21:16Z)
• AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild [62.47761809929869]
  Vision-language navigation (VLN) requires intelligent agents to navigate environments by interpreting linguistic instructions alongside visual observations.<n>Current VLN research for unmanned aerial vehicles (UAVs) relies on detailed, pre-specified instructions to guide the UAV along predetermined routes.<n>We propose AutoFly, an end-to-end Vision-Language-Action model for autonomous UAV navigation.
  arXiv  Detail & Related papers  (2026-02-10T11:08:07Z)
• Percept-WAM: Perception-Enhanced World-Awareness-Action Model for Robust End-to-End Autonomous Driving [48.512353531499286]
  We introduce Percept-WAM, a perception-enhanced World-Awareness-Action Model that implicitly integrates 2D/3D scene understanding abilities within a single vision-language model (VLM)<n>We propose a grid-conditioned prediction mechanism for dense object perception, incorporating IoU-aware scoring and parallel autoregressive decoding, improving stability in long-tail, far-range, and small-object scenarios.<n>Experiments show that Percept-WAM matches or surpasses classical detectors and segmenters on downstream perception benchmarks, achieving 51.7/58.9 mAP on 2D detection and nuScenes BEV 3D detection
  arXiv  Detail & Related papers  (2025-11-24T15:28:25Z)
• SpaRC-AD: A Baseline for Radar-Camera Fusion in End-to-End Autonomous Driving [5.343552118560704]
  SpaRC-AD is a query-based end-to-end camera-radar fusion framework for planning-oriented autonomous driving.<n>Our method achieves strong improvements over the state-of-the-art vision-only baselines across multiple autonomous driving tasks.
  arXiv  Detail & Related papers  (2025-08-14T12:02:41Z)
• Natural Reflection Backdoor Attack on Vision Language Model for Autonomous Driving [55.96227460521096]
  Vision-Language Models (VLMs) have been integrated into autonomous driving systems to enhance reasoning capabilities.<n>We propose a natural reflection-based backdoor attack targeting VLM systems in autonomous driving scenarios.<n>Our findings uncover a new class of attacks that exploit the stringent real-time requirements of autonomous driving.
  arXiv  Detail & Related papers  (2025-05-09T20:28:17Z)
• Multi-camera Bird's Eye View Perception for Autonomous Driving [17.834495597639805]
  It is essential to produce perception outputs in 3D to enable the spatial reasoning of other agents and structures. The most basic approach to achieving the desired BEV representation from a camera image is IPM, assuming a flat ground surface. More recent approaches use deep neural networks to output directly in BEV space.
  arXiv  Detail & Related papers  (2023-09-16T19:12:05Z)
• VILENS: Visual, Inertial, Lidar, and Leg Odometry for All-Terrain Legged Robots [5.789654849162465]
  We present visual inertial lidar legged navigation system (VILENS) for legged robots. The key novelty is the tight fusion of four different sensor modalities to achieve reliable operation. We show an average improvement of 62% translational and 51% rotational errors compared to a state-of-the-art loosely coupled approach.
  arXiv  Detail & Related papers  (2021-07-15T11:05:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.