LoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon Embodied Tasks

• URL: http://arxiv.org/abs/2506.00411v1
• Date: Sat, 31 May 2025 06:01:03 GMT
• Title: LoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon Embodied Tasks
• Abstract summary: Real-world embodied agents face high-level goals demanding multi-step solutions.<n>Long-horizon tasks require high-level task planning and low-level motion control.<n>We introduce a new unified vision language framework for long-horizon tasks, dubbed LoHoVLA.
• Score: 31.3295171851909
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world embodied agents face long-horizon tasks, characterized by high-level goals demanding multi-step solutions beyond single actions. Successfully navigating these requires both high-level task planning (i.e., decomposing goals into sub-tasks) and low-level motion control (i.e., generating precise robot actions). While existing vision language action (VLA) models and hierarchical architectures offer potential in embodied tasks, the former often falter in planning, and the latter can suffer from coordination issues, both hampering performance. We introduce a new unified VLA framework for long-horizon tasks, dubbed LoHoVLA, to overcome these limitations. LoHoVLA leverages a large pretrained vision language model (VLM) as the backbone to jointly generate language and action tokens for sub-task generation and robot action prediction, respectively. This shared representation promotes better generalization across tasks. Additionally, LoHoVLA embraces a hierarchical closed-loop control mechanism to mitigate errors originating from both high-level planning and low-level control. To train LoHoVLA, we introduce LoHoSet, a dataset built on the Ravens simulator, containing 20 long-horizon tasks, each with 1,000 expert demonstrations composed of visual observations, linguistic goals, sub-tasks, and robot actions. Experimental results show that LoHoVLA significantly surpasses both hierarchical and standard VLA approaches on long-horizon embodied tasks in the Ravens simulator. These findings underscore the promise of unified architectures for advancing generalizable embodied intelligence.

Related papers

• Beyond Flat Policies: Hierarchical Post-Training for Embodied Agents in Robotic Manipulation [34.87301719899886]
  Vision-language-action (VLA) models have demonstrated remarkable capabilities in robotic manipulation by leveraging pretrained vision-language models.<n>Existing post-training methods predominantly optimize VLA models as flat policies, making it difficult to explicitly model task progression and perform robust long-horizon manipulation.<n>We propose Hierarchical Robotic Control (HiRoC), a hierarchical post-training framework that decouples high-level task planning from low-level action execution.
  arXiv  Detail & Related papers  (2026-08-06T13:07:56Z)
• Anticipation-VLA: Solving Long-Horizon Embodied Tasks via Anticipation-based Subgoal Generation [23.021987362787225]
  Vision-Language-Action (VLA) models have emerged as a powerful paradigm for embodied intelligence.<n>We propose Anticipation-VLA, a hierarchical VLA model that leverages the anticipation model to generate actionable subgoals.<n>We implement Anticipation-VLA with finetuning a Unified Multimodal Model (UMM) for high-level subgoal generation and a goal-conditioned VLA policy for low-level action execution.
  arXiv  Detail & Related papers  (2026-05-03T08:11:35Z)
• HiVLA: A Visual-Grounded-Centric Hierarchical Embodied Manipulation System [48.02804864175127]
  HiVLA is a visual-grounded-centric hierarchical framework that explicitly decouples high-level semantic planning from low-level motor control.<n>Experiments in simulation and the real world demonstrate that HiVLA significantly outperforms state-of-the-art end-to-end baselines.
  arXiv  Detail & Related papers  (2026-04-15T17:50:07Z)
• LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policies [54.150202739999806]
  LiLo-VLA is a modular framework capable of zero-shot modularity to novel long-horizon tasks without ever being trained on them.<n>We introduce a 21-task simulation benchmark consisting of two challenging suites: LIBERO-Long++ and Ultra-Long.<n>In these simulations, LiLo-VLA achieves a 69% average success rate, outperforming Pi0.5 by 41% and OpenVLA-OFT by 67%.
  arXiv  Detail & Related papers  (2026-02-25T03:33:39Z)
• Scaling World Model for Hierarchical Manipulation Policies [61.736772957803026]
  Vision-Language-Action (VLA) models are promising for generalist robot manipulation but brittle in out-of-distribution settings.<n>We introduce a hierarchical Vision-Language-Action framework our that leverages the generalization of large-scale pre-trained world model.<n>We validate both visual goal synthesis and our hierarchical VLA policies in massive out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2026-02-11T16:12:33Z)
• H-AIM: Orchestrating LLMs, PDDL, and Behavior Trees for Hierarchical Multi-Robot Planning [3.2800662172795114]
  H-AIM is a novel embodied multi-robot task planning framework.<n>It exploits large language models (LLMs) to parse instructions and generate Planning Domain Definition Language (PDDL) problem descriptions.<n>It compiles the resulting plan into behavior trees for reactive control.
  arXiv  Detail & Related papers  (2026-01-16T07:59:50Z)
• LoLA: Long Horizon Latent Action Learning for General Robot Manipulation [38.209790870296835]
  Long Horizon Latent Action Learning (LoLA) is a framework designed for robot manipulation.<n>We first employ Vision-Language Models to encode rich contextual features from historical sequences and multi-view observations.<n>We then introduce State-Aware Latent Re-representation, which transforms visual inputs and language commands into actionable robot motion space.
  arXiv  Detail & Related papers  (2025-12-23T08:45:24Z)
• dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thought [66.78110237549087]
  Vision-Language-Action (VLA) models are emerging as a next-generation paradigm for robotics.<n>We introduce dVLA, a diffusion-based VLA that unifies visual perception, language reasoning, and robotic control in a single system.
  arXiv  Detail & Related papers  (2025-09-30T02:36:11Z)
• From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models [5.660635614478238]
  Vision-Language-Action (VLA) models promise to produce versatile, "generalist" robot policies.<n>Traditional imitation learning benchmarks are unsuitable due to the lack of language instructions.<n>We introduce a unified suite of 50 simulation-based tasks across 10 subcategories spanning language instruction, vision, and objects.
  arXiv  Detail & Related papers  (2025-06-11T16:52:18Z)
• RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation [80.20970723577818]
  We introduce RoboCerebra, a benchmark for evaluating high-level reasoning in long-horizon robotic manipulation.<n>The dataset is constructed via a top-down pipeline, where GPT generates task instructions and decomposes them into subtask sequences.<n>Compared to prior benchmarks, RoboCerebra features significantly longer action sequences and denser annotations.
  arXiv  Detail & Related papers  (2025-06-07T06:15:49Z)
• CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models [89.44024245194315]
  We introduce a method that incorporates explicit visual chain-of-thought (CoT) reasoning into vision-language-action models (VLAs)<n>We introduce CoT-VLA, a state-of-the-art 7B VLA that can understand and generate visual and action tokens.<n>Our experimental results demonstrate that CoT-VLA achieves strong performance, outperforming the state-of-the-art VLA model by 17% in real-world manipulation tasks and 6% in simulation benchmarks.
  arXiv  Detail & Related papers  (2025-03-27T22:23:04Z)
• Data-Agnostic Robotic Long-Horizon Manipulation with Vision-Language-Guided Closed-Loop Feedback [12.600525101342026]
  We introduce DAHLIA, a data-agnostic framework for language-conditioned long-horizon robotic manipulation.<n>LLMs are large language models for real-time task planning and execution.<n>Our framework demonstrates state-of-the-art performance across diverse long-horizon tasks, achieving strong generalization in both simulated and real-world scenarios.
  arXiv  Detail & Related papers  (2025-03-27T20:32:58Z)
• CHOP: Mobile Operating Assistant with Constrained High-frequency Optimized Subtask Planning [18.826366389246385]
  We propose a new mobile assistant architecture with constrained high-frequency optimized planning (CHOP)<n>Our approach overcomes the VLM's deficiency in GUI scenarios planning by using human-planned subtasks as the basis vector.<n>We evaluate our architecture in both English and Chinese contexts across 20 Apps, demonstrating significant improvements in both effectiveness and efficiency.
  arXiv  Detail & Related papers  (2025-03-05T18:56:16Z)
• VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks [100.3234156027118]
  We present VLABench, an open-source benchmark for evaluating universal LCM task learning.<n>VLABench provides 100 carefully designed categories of tasks, with strong randomization in each category of task and a total of 2000+ objects.<n>The benchmark assesses multiple competencies including understanding of mesh&texture, spatial relationship, semantic instruction, physical laws, knowledge transfer and reasoning.
  arXiv  Detail & Related papers  (2024-12-24T06:03:42Z)
• A Dual Process VLA: Efficient Robotic Manipulation Leveraging VLM [0.26334346517416873]
  Vision-Language-Action (VLA) models enable robots to perform complex tasks by integrating visual context with linguistic commands. To overcome this, we propose Dual Process VLA (DP-VLA), a hierarchical framework inspired by dual-process theory. Experimental results on the RoboCasa dataset demonstrate that DP-VLA achieves faster inference and higher task success rates.
  arXiv  Detail & Related papers  (2024-10-21T00:36:02Z)
• A Survey on Vision-Language-Action Models for Embodied AI [71.16123093739932]
  Embodied AI is widely recognized as a key element of artificial general intelligence.<n>A new category of multimodal models has emerged to address language-conditioned robotic tasks in embodied AI.<n>We present the first survey on vision-language-action models for embodied AI.
  arXiv  Detail & Related papers  (2024-05-23T01:43:54Z)
• Generalizable Long-Horizon Manipulations with Large Language Models [91.740084601715]
  This work introduces a framework harnessing the capabilities of Large Language Models (LLMs) to generate primitive task conditions for generalizable long-horizon manipulations. We create a challenging robotic manipulation task suite based on Pybullet for long-horizon task evaluation.
  arXiv  Detail & Related papers  (2023-10-03T17:59:46Z)
• GRILL: Grounded Vision-language Pre-training via Aligning Text and Image Regions [92.96783800362886]
  Generalization to unseen tasks is an important ability for few-shot learners to achieve better zero-/few-shot performance on diverse tasks. We introduce GRILL, a novel VL model that can be generalized to diverse tasks including visual question answering, captioning, and grounding tasks with no or very few training instances.
  arXiv  Detail & Related papers  (2023-05-24T03:33:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.