CodeGraphVLP: Code-as-Planner Meets Semantic-Graph State for Non-Markovian Vision-Language-Action Models

• URL: http://arxiv.org/abs/2604.22238v1
• Date: Fri, 24 Apr 2026 05:27:27 GMT
• Title: CodeGraphVLP: Code-as-Planner Meets Semantic-Graph State for Non-Markovian Vision-Language-Action Models
• Abstract summary: Vision-Language-Action (VLA) models promise generalist robot manipulation, but are typically trained and deployed as short-horizon policies.<n>We present CodeGraph, a hierarchical framework that enables reliable long-horizon manipulation.
• Score: 13.691036198349465
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action (VLA) models promise generalist robot manipulation, but are typically trained and deployed as short-horizon policies that assume the latest observation is sufficient for action reasoning. This assumption breaks in non-Markovian long-horizon tasks, where task-relevant evidence can be occluded or appear only earlier in the trajectory, and where clutter and distractors make fine-grained visual grounding brittle. We present CodeGraphVLP, a hierarchical framework that enables reliable long-horizon manipulation by combining a persistent semantic-graph state with an executable code-based planner and progress-guided visual-language prompting. The semantic-graph maintains task-relevant entities and relations under partial observability. The synthesized planner executes over this semantic-graph to perform efficient progress checks and outputs a subtask instruction together with subtask-relevant objects. We use these outputs to construct clutter-suppressed observations that focus the VLA executor on critical evidence. On real-world non-Markovian tasks, CodeGraphVLP improves task completion over strong VLA baselines and history-enabled variants while substantially lowering planning latency compared to VLM-in-the-loop planning. We also conduct extensive ablation studies to confirm the contributions of each component.

Related papers

• Towards Neuro-Symbolic Procedural Reasoning for Long-Horizon Vision-Language-Action Manipulation [12.461719086222788]
  Vision-language-action (VLA) models can execute short manipulation skills, but remain brittle in long-horizon procedures.<n>We investigate a neuro-symbolic framework that combines learned VLA control with explicit task graphs and multimodal procedural memory.
  arXiv  Detail & Related papers  (2026-09-04T17:14:50Z)
• Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance [108.46436073194546]
  We argue that effective planning should be local, visually grounded, internally generated, and directly aligned with action.<n>We propose Afford-VLA, a unified framework that internalizes task-conditioned affordance as an explicit visual planning interface within VLA models.
  arXiv  Detail & Related papers  (2026-05-22T20:43:47Z)
• Long-Horizon Manipulation via Trace-Conditioned VLA Planning [55.80061850746898]
  LoHo-Manip is a framework that scales short-horizon VLA execution to long-horizon instruction following via a dedicated task-management VLM.<n>LoHo-Manip predicts a progress-aware remaining plan that combines a subtask sequence with an explicit done + remaining split as lightweight language memory.<n>The executor VLA is adapted to condition on the rendered trace, thereby turning long-horizon decision-making into repeated local control.
  arXiv  Detail & Related papers  (2026-04-23T17:59:04Z)
• V-CAGE: Context-Aware Generation and Verification for Scalable Long-Horizon Embodied Tasks [6.820118518027692]
  V-CAGE is a closed-loop framework for generating semantically aligned manipulation datasets at scale.<n>We propose a context-aware instantiation mechanism that enforces geometric consistency during scene synthesis.<n>We also employ a hierarchical instruction decomposition module to bridge the gap between abstract intent and low-level control.
  arXiv  Detail & Related papers  (2026-01-21T16:41:51Z)
• RDD: Retrieval-Based Demonstration Decomposer for Planner Alignment in Long-Horizon Tasks [21.341051218915535]
  We propose a Demonstration Decomposer that automatically decomposes demonstrations into sub-tasks.<n>Our method outperforms the state-of-the-art sub-task decomposer on both simulation and real-world tasks.
  arXiv  Detail & Related papers  (2025-10-16T17:59:37Z)
• Plan-over-Graph: Towards Parallelable LLM Agent Schedule [53.834646147919436]
  Large Language Models (LLMs) have demonstrated exceptional abilities in reasoning for task planning.<n>This paper introduces a novel paradigm, plan-over-graph, in which the model first decomposes a real-life textual task into executable subtasks and constructs an abstract task graph.<n>The model then understands this task graph as input and generates a plan for parallel execution.
  arXiv  Detail & Related papers  (2025-02-20T13:47:51Z)
• Instance-Aware Graph Prompt Learning [71.26108600288308]
  We introduce Instance-Aware Graph Prompt Learning (IA-GPL) in this paper. The process involves generating intermediate prompts for each instance using a lightweight architecture. Experiments conducted on multiple datasets and settings showcase the superior performance of IA-GPL compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-11-26T18:38:38Z)
• VeriGraph: Scene Graphs for Execution Verifiable Robot Planning [33.8868315479384]
  We propose VeriGraph, a framework that integrates vision-language models (VLMs) for robotic planning while verifying action feasibility. VeriGraph employs scene graphs as an intermediate representation, capturing key objects and spatial relationships to improve plan verification and refinement. Our approach significantly enhances task completion rates across diverse manipulation scenarios, outperforming baseline methods by 58% for language-based tasks and 30% for image-based tasks.
  arXiv  Detail & Related papers  (2024-11-15T18:59:51Z)
• Towards Unified Token Learning for Vision-Language Tracking [65.96561538356315]
  We present a vision-language (VL) tracking pipeline, termed textbfMMTrack, which casts VL tracking as a token generation task. Our proposed framework serializes language description and bounding box into a sequence of discrete tokens. In this new design paradigm, all token queries are required to perceive the desired target and directly predict spatial coordinates of the target.
  arXiv  Detail & Related papers  (2023-08-27T13:17:34Z)
• Unsupervised Task Graph Generation from Instructional Video Transcripts [53.54435048879365]
  We consider a setting where text transcripts of instructional videos performing a real-world activity are provided. The goal is to identify the key steps relevant to the task as well as the dependency relationship between these key steps. We propose a novel task graph generation approach that combines the reasoning capabilities of instruction-tuned language models along with clustering and ranking components.
  arXiv  Detail & Related papers  (2023-02-17T22:50:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.