LiteVLA-H: Dual-Rate Vision-Language-Action Inference for Onboard Aerial Guidance and Semantic Perception

• URL: http://arxiv.org/abs/2605.00884v1
• Date: Mon, 27 Apr 2026 06:42:50 GMT
• Title: LiteVLA-H: Dual-Rate Vision-Language-Action Inference for Onboard Aerial Guidance and Semantic Perception
• Abstract summary: We present LiteVLA-H, a compact 256M- parameter VLA system for dual-rate operation on an Jets NVIDIAon AGX Orin.<n>The central empirical observation is that, in this compact edge regime, end-to-end latency is dominated by multimodal pre-fill.<n>To specialize the model without collapsing its descriptive competence, we use a knowledge-preserving fine-tuning recipe that mixes reactive flight data, aerial semantic data, and generic caption/VQA supervision.
• Score: 0.6119773373677944
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language-action (VLA) models have shown strong semantic grounding and task generalization in manipulation, but aerial deployment remains difficult because drones require low-latency closed-loop guidance under strict onboard compute and communication constraints. We present LiteVLA-H, a compact 256M-parameter VLA system designed for dual-rate operation on an NVIDIA Jetson AGX Orin: a fast outer-loop guidance mode for short action-token outputs and a slower semantic mode for scene understanding, hazard description, and operator-facing narration. The central empirical observation is that, in this compact edge regime, end-to-end latency is dominated by multimodal pre-fill rather than by the marginal cost of decoding a few extra tokens. This motivates a scheduler that issues reactive action tokens at 50.65,ms (19.74,Hz) while still supporting sentence-level semantic outputs at 149.90--164.57\ms (6.08--6.67,Hz) on the same embedded platform. To specialize the model without collapsing its descriptive competence, we use a knowledge-preserving fine-tuning recipe that mixes reactive flight data, aerial semantic data, and generic caption/VQA supervision. Beyond reporting current latency measurements, we position the system against recent state-of-the-art architectures, including AnywhereVLA, FutureVLA, and ReMem-VLA, showing that the measured action branch reaches a higher edge inference rate under our deployment conditions while retaining periodic semantic awareness.

Related papers

• Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verification [24.0529851878964]
  Vision-Language-Action (VLA) models have demonstrated remarkable capabilities in the field of embodied AI, but their high computational cost and limited predicted action length hinder real-time deployment.<n>We propose SpecVLA, an algorithm-system co-design framework that adaptively balances action length, inference latency, and task reliability.
  arXiv  Detail & Related papers  (2026-08-16T08:59:22Z)
• S-squared-VLA: Decoupling Semantic and Spatial Streams in Vision-Language-Action Models for Autonomous Driving [8.813976817688605]
  Vision-Language Models (VLMs) have demonstrated remarkable potential for high-level reasoning in autonomous driving.<n>VLMs struggle to generate precise, low-level control actions due to mismatch between discrete language tokens and continuous trajectory planning.<n>We propose the S-squared-VLA, which explicitly decouples the semantic and spatial streams in Vision-Language-Action models.
  arXiv  Detail & Related papers  (2026-07-15T15:07:02Z)
• VisualThink-VLA: Visual Intermediate Reasoning for Effective and Low-Latency Vision-Language-Action Policies [58.65913948991329]
  We present VISUALTHINK-VLA, a visual intermediate-reasoning framework for accurate, low-latency VLA policies.<n>Our bootstrapping philosophy is to guide action with effective visual thinking.<n>It bootstraps action prediction through a compact visual-evidence interface that preserves spatial precision while avoiding decoding overhead.
  arXiv  Detail & Related papers  (2026-05-28T14:36:53Z)
• QuoVLA: Quotient Space for Vision-Language-Action Models [51.02329790939691]
  Vision-Language-Action (VLA) models commonly adapt pretrained VLM latents to robot control by mapping visual observations and language instructions to continuous actions.<n>Our textitQuotient Theory for VLA shows that pretrained VLM latents are not action-insufficient but action-sufficient.<n>We propose QuoVLA, a quotient-space framework for VLA that compresses pretrained VLM latents into action-sufficient representations.
  arXiv  Detail & Related papers  (2026-05-24T06:28:53Z)
• Sentinel-VLA: A Metacognitive VLA Model with Active Status Monitoring for Dynamic Reasoning and Error Recovery [62.75419724651416]
  We introduce textbfSentinel-VLA, a metacognitive VLA model equipped with an active sentinel'' module to monitor real-time execution status.<n>All training data is automatically generated and annotated through our designed pipeline.<n>Real-world experiments demonstrate that Sentinel-VLA boosts the task success rate by over 30% compared to the SOTA model, PI0.
  arXiv  Detail & Related papers  (2026-05-02T02:10:54Z)
• VP-VLA: Visual Prompting as an Interface for Vision-Language-Action Models [80.1055544841585]
  Vision-Language-Action models typically map visual observations and linguistic instructions directly to robotic control signals.<n>We propose VP-VLA, a dual-system framework that decouples high-level reasoning and low-level execution via a structured visual prompting interface.<n> Experiments on the Robocasa-GR1-Tabletop benchmark and SimplerEnv simulation demonstrate that VP-VLA improves success rates by 5% and 8.3%.
  arXiv  Detail & Related papers  (2026-03-23T14:08:58Z)
• StreamVLA: Breaking the Reason-Act Cycle via Completion-State Gating [8.031311182554981]
  Long-horizon robotic manipulation requires bridging the gap between high-level planning and low-level control.<n>Current Vision-Language-Action models often entangle these processes, performing redundant multimodal reasoning at every timestep.<n>We present StreamVLA, a dual-system architecture that unifies textual task decomposition, visual goal imagination, and continuous action generation within a single parameter-efficient backbone.
  arXiv  Detail & Related papers  (2026-02-01T08:51:17Z)
• TIDAL: Temporally Interleaved Diffusion and Action Loop for High-Frequency VLA Control [15.534182843429043]
  Large-scale Vision-Language-Action (VLA) models offer semantic generalization but suffer from high inference latency.<n>We propose TIDAL, a hierarchical framework that decouples semantic reasoning from high-frequency actuation.<n> TIDAL operates as a backbone-agnostic module for diffusion-basedVLAs, using a dual-frequency architecture.
  arXiv  Detail & Related papers  (2026-01-21T12:43:11Z)
• Token Expand-Merge: Training-Free Token Compression for Vision-Language-Action Models [16.321608201919474]
  Vision-Language-Action (VLA) models pretrained on large-scale multimodal datasets have emerged as powerful foundations for robotic perception and control.<n>We propose Expand Token-and-Merge-VLA, a training-free token compression framework that accelerates VLA inference while preserving task performance.
  arXiv  Detail & Related papers  (2025-12-10T18:59:24Z)
• SemanticVLA: Semantic-Aligned Sparsification and Enhancement for Efficient Robotic Manipulation [65.6201974979119]
  We propose SemanticVLA, a novel VLA framework that performs Semantic-Hierarchical Sparsification and Enhancement for Efficient Robotic Manipulation.<n>SemanticVLA surpasses OpenVLA on LIBERO benchmark by 21.1% in success rate, while reducing training cost and inference latency by 3.0-fold and 2.7-fold.
  arXiv  Detail & Related papers  (2025-11-13T17:24:37Z)
• Open-Vocabulary Spatio-Temporal Scene Graph for Robot Perception and Teleoperation Planning [55.90805559207812]
  In dynamic remote scenes, transmission latency during bidirectional communication creates gaps between remote perceived states and operator intent.<n>We present a representation that enriches open-vocabulary perception with temporal dynamics and lightweight latency annotations.<n>We show that our method achieves 74 percent node accuracy on the Replica benchmark, outperforming Concept.Graph.
  arXiv  Detail & Related papers  (2025-09-27T04:31:24Z)
• NinA: Normalizing Flows in Action. Training VLA Models with Normalizing Flows [75.70583906344815]
  Diffusion models have been widely adopted as action decoders due to their ability to model complex, multimodal action distributions.<n>We present NinA, a fast and expressive alternative to diffusion-based decoders for Vision-Language-Action (VLA) models.
  arXiv  Detail & Related papers  (2025-08-23T00:02:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.