UniPlan: Vision-Language Task Planning for Mobile Manipulation with Unified PDDL Formulation

• URL: http://arxiv.org/abs/2602.08537v1
• Date: Mon, 09 Feb 2026 11:35:21 GMT
• Title: UniPlan: Vision-Language Task Planning for Mobile Manipulation with Unified PDDL Formulation
• Abstract summary: UniPlan is a vision task planning system for mobilemanipulation in largescale indoor environments.<n>It unifies scene topology, visuals, and robot capabilities into a holistic PDDL representation.<n>It significantly outperforms VLM and LLM+DL planning success rate, plan quality, and computational efficiency.
• Score: 44.71201089970223
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Integration of VLM reasoning with symbolic planning has proven to be a promising approach to real-world robot task planning. Existing work like UniDomain effectively learns symbolic manipulation domains from real-world demonstrations, described in Planning Domain Definition Language (PDDL), and has successfully applied them to real-world tasks. These domains, however, are restricted to tabletop manipulation. We propose UniPlan, a vision-language task planning system for long-horizon mobile-manipulation in large-scale indoor environments, that unifies scene topology, visuals, and robot capabilities into a holistic PDDL representation. UniPlan programmatically extends learned tabletop domains from UniDomain to support navigation, door traversal, and bimanual coordination. It operates on a visual-topological map, comprising navigation landmarks anchored with scene images. Given a language instruction, UniPlan retrieves task-relevant nodes from the map and uses a VLM to ground the anchored image into task-relevant objects and their PDDL states; next, it reconnects these nodes to a compressed, densely-connected topological map, also represented in PDDL, with connectivity and costs derived from the original map; Finally, a mobile-manipulation plan is generated using off-the-shelf PDDL solvers. Evaluated on human-raised tasks in a large-scale map with real-world imagery, UniPlan significantly outperforms VLM and LLM+PDDL planning in success rate, plan quality, and computational efficiency.

Related papers

• Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance [108.46436073194546]
  We argue that effective planning should be local, visually grounded, internally generated, and directly aligned with action.<n>We propose Afford-VLA, a unified framework that internalizes task-conditioned affordance as an explicit visual planning interface within VLA models.
  arXiv  Detail & Related papers  (2026-05-22T20:43:47Z)
• Scale-Plan: Scalable Language-Enabled Task Planning for Heterogeneous Multi-Robot Teams [11.964765256346857]
  Long-horizon task planning for heterogeneous multi-robot systems is essential for deploying collaborative teams in real-world environments.<n>We present Scale-Plan, a scalable LLM-assisted framework that generates compact, task-relevant problem representations from natural language instructions.<n>By filtering irrelevant information prior to planning, Scale-Plan enables efficient decomposition, allocation, and long-horizon plan generation.
  arXiv  Detail & Related papers  (2026-03-09T18:13:18Z)
• PSALM-V: Automating Symbolic Planning in Interactive Visual Environments with Large Language Models [22.688086293676328]
  We propose PSALM-V, the first autonomous neuro-symbolic learning system able to induce symbolic action semantics (i.e., pre- and post-conditions) in visual environments.<n> PSALM-V bootstraps reliable symbolic planning without expert action definitions, using LLMs to generate plans and candidate symbolic semantics.
  arXiv  Detail & Related papers  (2025-06-25T02:44:20Z)
• Gondola: Grounded Vision Language Planning for Generalizable Robotic Manipulation [62.711546725154314]
  We introduce Gondola, a grounded vision-language planning model based on large language models (LLMs) for generalizable robotic manipulation.<n>G Gondola takes multi-view images and history plans to produce the next action plan with interleaved texts and segmentation masks of target objects and locations.<n>G Gondola outperforms the state-of-the-art LLM-based method across all four levels of the GemBench dataset.
  arXiv  Detail & Related papers  (2025-06-12T20:04:31Z)
• Planning with Vision-Language Models and a Use Case in Robot-Assisted Teaching [0.9217021281095907]
  This paper introduces Image2PDDL, a novel framework that leverages Vision-Language Models (VLMs) to automatically convert images of initial states and descriptions of goal states into PDDL problems.<n>We evaluate the framework on various domains, including standard planning domains like blocksworld and sliding tile puzzles, using datasets with multiple difficulty levels.<n>We will discuss a potential use case in robot-assisted teaching of students with Autism Spectrum Disorder.
  arXiv  Detail & Related papers  (2025-01-29T14:04:54Z)
• DKPROMPT: Domain Knowledge Prompting Vision-Language Models for Open-World Planning [9.31108717722043]
  Vision-language models (VLMs) have been applied to robot task planning problems. DKPROMPT automates VLM prompting using domain knowledge in PDDL for classical planning in open worlds.
  arXiv  Detail & Related papers  (2024-06-25T15:49:47Z)
• PDDLEGO: Iterative Planning in Textual Environments [56.12148805913657]
  Planning in textual environments has been shown to be a long-standing challenge even for current models. We propose PDDLEGO that iteratively construct a planning representation that can lead to a partial plan for a given sub-goal. We show that plans produced by few-shot PDDLEGO are 43% more efficient than generating plans end-to-end on the Coin Collector simulation.
  arXiv  Detail & Related papers  (2024-05-30T08:01:20Z)
• SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning [15.346150968195015]
  We introduce SayPlan, a scalable approach to large-scale task planning for robotics using 3D scene graph (3DSG) representations. We evaluate our approach on two large-scale environments spanning up to 3 floors and 36 rooms with 140 assets and objects.
  arXiv  Detail & Related papers  (2023-07-12T12:37:55Z)
• Long-Horizon Planning and Execution with Functional Object-Oriented Networks [79.94575713911189]
  We introduce the idea of exploiting object-level knowledge as a FOON for task planning and execution. Our approach automatically transforms FOON into PDDL and leverages off-the-shelf planners, action contexts, and robot skills. We demonstrate our approach on long-horizon tasks in CoppeliaSim and show how learned action contexts can be extended to never-before-seen scenarios.
  arXiv  Detail & Related papers  (2022-07-12T19:29:35Z)
• Online Grounding of PDDL Domains by Acting and Sensing in Unknown Environments [62.11612385360421]
  This paper proposes a framework that allows an agent to perform different tasks. We integrate machine learning models to abstract the sensory data, symbolic planning for goal achievement and path planning for navigation. We evaluate the proposed method in accurate simulated environments, where the sensors are RGB-D on-board camera, GPS and compass.
  arXiv  Detail & Related papers  (2021-12-18T21:48:20Z)
• Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulation [74.88956115580388]
  Planning is performed in a low-dimensional latent state space that embeds images. Our framework consists of two main components: a Visual Foresight Module (VFM) that generates a visual plan as a sequence of images, and an Action Proposal Network (APN) that predicts the actions between them.
  arXiv  Detail & Related papers  (2020-03-19T18:43:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.