Towards Neuro-Symbolic Procedural Reasoning for Long-Horizon Vision-Language-Action Manipulation

• URL: http://arxiv.org/abs/2609.05369v1
• Date: Fri, 04 Sep 2026 17:14:50 GMT
• Title: Towards Neuro-Symbolic Procedural Reasoning for Long-Horizon Vision-Language-Action Manipulation
• Abstract summary: Vision-language-action (VLA) models can execute short manipulation skills, but remain brittle in long-horizon procedures.<n>We investigate a neuro-symbolic framework that combines learned VLA control with explicit task graphs and multimodal procedural memory.
• Score: 12.461719086222788
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language-action (VLA) models can execute short manipulation skills, but remain brittle in long-horizon procedures requiring persistent task state, dependency-aware reasoning, conditional decisions, and reliable grounding. We investigate a neuro-symbolic framework that combines learned VLA control with explicit task graphs and multimodal procedural memory. Task graphs encode action dependencies, valid transitions, and branch conditions, while memory maintains the active step, completed actions, textual context, and task-relevant visual evidence. Together, these structures guide object selection, destination grounding, subgoal dispatch, and verification of expected state transitions. Human demonstrations provide additional spatial and temporal guidance through gaze or saliency cues. To isolate their effect on policy learning, our initial study bypasses cross-view gaze transfer and directly annotates pseudo-gaze in robot-view teleoperation videos. The resulting guidance is used during VLA fine-tuning and inference. We study two long-horizon manipulation domains, workspace clearing and surgical-instrument handling, which require ordered execution, visually grounded decisions, and conditional branching. We evaluate correct-object and destination selection, subtask completion, task progress, step-order consistency, complete-task success, and procedural or execution mistakes. This work positions structured symbolic reasoning and demonstration-derived visual guidance as complementary mechanisms for reliable long-horizon VLA manipulation.

Related papers

• HINT: Human-Intent Inception for Long-Horizon Robot Manipulation [11.55318573740128]
  We present HINT (Human-INTent INcepTion), an agentic framework inspired by the human manipulation principles.<n>We show that HINT substantially improves intent understanding, task progress, and end-to-end success across two foundation policies.
  arXiv  Detail & Related papers  (2026-09-02T14:26:30Z)
• WorldScape Policy 2.0: Empowering Steerable World Action Modeling with Reasoning-Augmented Memory [50.19449886428936]
  We introduce WorldScape Policy 2.0, a controllable WAM with reasoning-augmented long short-term memory.<n>The retrieved history augments perception and autoregressively generated planning tokens, yielding an implicit subgoal condition for autonomous planning.<n>Experiments in both simulation and real-world platforms demonstrate superior capabilities in long-horizon autonomous planning, fine-grained instruction following, and in-context adaptation.
  arXiv  Detail & Related papers  (2026-07-21T08:25:37Z)
• PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations [66.94988600664574]
  We present textbfPRTS (textbfPrimitive textbfReasoning and textbfTasking textbfSystem), a VLA foundation model that reformulates pretraining through Goal-Conditioned Reinforcement Learning.
  arXiv  Detail & Related papers  (2026-04-30T06:14:02Z)
• CodeGraphVLP: Code-as-Planner Meets Semantic-Graph State for Non-Markovian Vision-Language-Action Models [13.691036198349465]
  Vision-Language-Action (VLA) models promise generalist robot manipulation, but are typically trained and deployed as short-horizon policies.<n>We present CodeGraph, a hierarchical framework that enables reliable long-horizon manipulation.
  arXiv  Detail & Related papers  (2026-04-24T05:27:27Z)
• Event-Driven Proactive Assistive Manipulation with Grounded Vision-Language Planning [14.022917047438375]
  We introduce a shift from request-driven assistance to event-driven proactive assistance.<n>We propose an event-driven framework that tracks interaction progress with an event monitor.<n>We evaluate the framework on a real tabletop number-block collaboration task.
  arXiv  Detail & Related papers  (2026-03-25T05:14:04Z)
• Non-Markovian Long-Horizon Robot Manipulation via Keyframe Chaining [56.62125584296097]
  Keyframe-Chaining VLA is a framework that extracts and links key historical frames to model long-horizon dependencies.<n>We design a progress-aware mechanism that dynamically retrieves historical frames based on their temporal relevance to the current execution phase.<n>We introduce a suite of four Non-Markovian manipulation tasks built upon the ManiSkill simulator to measure task success rates.
  arXiv  Detail & Related papers  (2026-03-02T05:26:29Z)
• CARE: Multi-Task Pretraining for Latent Continuous Action Representation in Robot Control [39.17038025776311]
  CARE is a framework designed to train VLA models for robotic task execution.<n> CARE eliminates the need for explicit action labels by leveraging only video-text pairs.<n>Results demonstrate CARE's scalability, interpretability, and effectiveness in robotic control with weak supervision.
  arXiv  Detail & Related papers  (2026-01-30T02:28:32Z)
• Action-Sketcher: From Reasoning to Action via Visual Sketches for Long-Horizon Robotic Manipulation [37.76888706583038]
  Long-horizon robotic manipulation is increasingly important for real-world deployment.<n>Existing end-to-end and hierarchical Vision-Language-Action policies often rely on text-only cues.<n>We introduce Visual Sketch, an implausible visual intermediate that renders points, boxes, arrows, and typed relations in the robot's current views.<n>We present Action-Sketcher, a VLA framework that operates in a cyclic See-Think-Sketch-Act workflow.
  arXiv  Detail & Related papers  (2026-01-04T17:53:42Z)
• Learning Affordances at Inference-Time for Vision-Language-Action Models [50.93181349331096]
  In robotics, Vision-Language-Action models (VLAs) offer a promising path towards solving complex control tasks.<n>We introduce Learning from Inference-Time Execution (LITEN), which connects a VLA low-level policy to a high-level VLM that conditions on past experiences.<n>Our approach iterates between a reasoning phase that generates and executes plans for the low-level VLA, and an assessment phase that reflects on the resulting execution.
  arXiv  Detail & Related papers  (2025-10-22T16:43:29Z)
• Exploring Conditions for Diffusion models in Robotic Control [70.27711404291573]
  We explore leveraging pre-trained text-to-image diffusion models to obtain task-adaptive visual representations for robotic control.<n>We find that naively applying textual conditions yields minimal or even negative gains in control tasks.<n>We propose ORCA, which introduces learnable task prompts that adapt to the control environment and visual prompts that capture fine-grained, frame-specific details.
  arXiv  Detail & Related papers  (2025-10-17T10:24:14Z)
• SAGE: Scene Graph-Aware Guidance and Execution for Long-Horizon Manipulation Tasks [3.688836621357062]
  Long-horizon manipulation tasks involve extended action sequences and complex object interactions.<n>We propose SAGE, a novel framework for Scene Graph-Aware Guidance and Execution in Long-Horizon Manipulation Tasks.<n> SAGE consists of two key components: (1) a scene graph-based task planner that uses VLMs and LLMs to parse the environment and reason about physically-grounded scene state transition sequences, and (2) a decoupled structural image editing pipeline that controllably converts each target sub-goal graph into a corresponding image.
  arXiv  Detail & Related papers  (2025-09-26T06:14:55Z)
• Modeling Long-horizon Tasks as Sequential Interaction Landscapes [75.5824586200507]
  We present a deep learning network that learns dependencies and transitions across subtasks solely from a set of demonstration videos. We show that these symbols can be learned and predicted directly from image observations. We evaluate our framework on two long horizon tasks: (1) block stacking of puzzle pieces being executed by humans, and (2) a robot manipulation task involving pick and place of objects and sliding a cabinet door with a 7-DoF robot arm.
  arXiv  Detail & Related papers  (2020-06-08T18:07:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.