PDDL-ART: Autonomous Symbolic Abstraction From Demonstration For Long-Horizon Robotic Manipulation Using Vision-Language Models

• URL: http://arxiv.org/abs/2608.17146v1
• Date: Mon, 17 Aug 2026 21:29:29 GMT
• Title: PDDL-ART: Autonomous Symbolic Abstraction From Demonstration For Long-Horizon Robotic Manipulation Using Vision-Language Models
• Abstract summary: We present a Vision-Language Model-based approach called PDDL-ART.<n>It generates task-specific PDDL domain and problem descriptions from a single expert demonstration.<n>PDDL-ART achieves an average success rate of 93.3%, compared to 78.3% for a baseline VLM-based planner.
• Score: 14.072904523937536
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Symbolic planning with PDDL offers a principled framework for long-horizon robot manipulation, but constructing accurate PDDL domain and problem descriptions remains a significant bottleneck, typically requiring substantial domain expertise. We present a Vision-Language Model (VLM)-based approach called PDDL-ART, a framework that autonomously generates task-specific PDDL domain and problem descriptions from a single expert demonstration, a natural language task description, and a library of available high-level action names. PDDL-ART does not require any domain templates, action signatures, or fine-tuning. To ensure the generated descriptions are not only syntactically valid but semantically aligned with the demonstrated task, PDDL-ART introduces a multi-stage correction pipeline operating at syntactic, semantic, and execution levels. A key component of execution-guided correction is symbolic predicate grounding. Instead of relying solely on visual observations, PDDL-ART leverages the tool-use capabilities of modern VLMs to incorporate geometric and temporal reasoning for evaluating relational predicates that are not directly discernible from images alone. Critically, the model autonomously determines when to invoke these tools and how to interpret their outputs. We evaluate PDDL-ART on challenging manipulation tasks in engine maintenance and household domains, including tasks that require memory, abstract predicate inference, and goal states that are visually indistinguishable from the initial state. PDDL-ART achieves an average success rate of 93.3%, compared to 78.3% for a baseline VLM-based planner.

Related papers

• Dynamic Parsing and Updating Natural Language Specification using VLMs for Robust Vision-Language Tracking [61.91472604294714]
  We propose a novel language dependency parsing mechanism to distill tracking principal components.<n>We exploit the powerful cross-modal understanding capability of the pre-trained vision-language model Qwen-VL.<n>Our method achieves consistent and superior tracking performance on multiple large-scale vision-language tracking benchmarks.
  arXiv  Detail & Related papers  (2026-06-28T12:12:18Z)
• PO-PDDL: Learning Symbolic POMDPs from Visual Demonstrations for Robot Planning Under Uncertainty [42.76829847298981]
  Real-world robot task planning must operate under both action execution and partial observability.<n>We introduce PO-PDDL, a symbolic formulation of Partially Observable Markov Decision Process (POMDP) models.<n>We show that our method consistently outperforms existing PDDL and POMDP model-learning approaches.
  arXiv  Detail & Related papers  (2026-06-14T07:46:28Z)
• AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding [55.535374902204296]
  AffordanceVLA is a unified framework that introduces structured affordance forecasting as a task-oriented intermediate representation.<n>We show that AffordanceVLA achieves strong performance across diverse manipulation scenarios.
  arXiv  Detail & Related papers  (2026-06-04T13:28:51Z)
• Compiling Temporal Numeric Planning into Discrete PDDL+: Extended Version [58.220343994928776]
  We present a practical compilation from temporal planning with durative actions into PDDL+.<n>Our compilation is experimentally shown to be of practical relevance for hard temporal numeric problems.
  arXiv  Detail & Related papers  (2026-03-12T17:19:30Z)
• Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future [125.92052530850425]
  Vision-Language-Action (VLA) frameworks integrate perception with language-grounded decision making.<n>VLA frameworks offer a pathway toward more interpretable, generalizable, and human-aligned driving policies.<n>This work aims to establish a coherent foundation for advancing human-compatible autonomous driving systems.
  arXiv  Detail & Related papers  (2025-12-18T16:57:44Z)
• PSALM-V: Automating Symbolic Planning in Interactive Visual Environments with Large Language Models [22.688086293676328]
  We propose PSALM-V, the first autonomous neuro-symbolic learning system able to induce symbolic action semantics (i.e., pre- and post-conditions) in visual environments.<n> PSALM-V bootstraps reliable symbolic planning without expert action definitions, using LLMs to generate plans and candidate symbolic semantics.
  arXiv  Detail & Related papers  (2025-06-25T02:44:20Z)
• Generating Symbolic World Models via Test-time Scaling of Large Language Models [28.258707611580643]
  Planning Domain Definition Language (PDDL) is leveraged as a planning abstraction that enables precise and formal state descriptions.<n>We introduce a simple yet effective algorithm, which first employs a Best-of-N sampling approach to improve the quality of the initial solution.<n>Our method outperforms o1-mini by a considerable margin in the generation of PDDL domains.
  arXiv  Detail & Related papers  (2025-02-07T07:52:25Z)
• Planning with Vision-Language Models and a Use Case in Robot-Assisted Teaching [0.9217021281095907]
  This paper introduces Image2PDDL, a novel framework that leverages Vision-Language Models (VLMs) to automatically convert images of initial states and descriptions of goal states into PDDL problems.<n>We evaluate the framework on various domains, including standard planning domains like blocksworld and sliding tile puzzles, using datasets with multiple difficulty levels.<n>We will discuss a potential use case in robot-assisted teaching of students with Autism Spectrum Disorder.
  arXiv  Detail & Related papers  (2025-01-29T14:04:54Z)
• PROC2PDDL: Open-Domain Planning Representations from Texts [56.627183903841164]
  Proc2PDDL is the first dataset containing open-domain procedural texts paired with expert-annotated PDDL representations. We show that Proc2PDDL is highly challenging, with GPT-3.5's success rate close to 0% and GPT-4's around 35%.
  arXiv  Detail & Related papers  (2024-02-29T19:40:25Z)
• Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning [39.29964085305846]
  Methods that use pre-trained large language models directly as planners are currently impractical due to limited correctness of plans. In this work, we introduce a novel alternative paradigm that constructs an explicit world (domain) model in planning domain definition language (PDDL) and then uses it to plan with sound domain-independent planners.
  arXiv  Detail & Related papers  (2023-05-24T08:59:15Z)
• PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models [127.17675443137064]
  We introduce PEVL, which enhances the pre-training and prompt tuning of vision-language models with explicit object position modeling. PEVL reformulates discretized object positions and language in a unified language modeling framework. We show that PEVL enables state-of-the-art performance on position-sensitive tasks such as referring expression comprehension and phrase grounding.
  arXiv  Detail & Related papers  (2022-05-23T10:17:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.