Cross-Domain Demo-to-Code via Neurosymbolic Counterfactual Reasoning

• URL: http://arxiv.org/abs/2603.18495v1
• Date: Thu, 19 Mar 2026 05:04:20 GMT
• Title: Cross-Domain Demo-to-Code via Neurosymbolic Counterfactual Reasoning
• Abstract summary: We formulate video-instructed robotic programming as a cross-domain adaptation problem.<n>NeSyCR is a neurosymbolic counterfactual reasoning framework.<n>NeSyCR achieves a 31.14% improvement in task success over the strongest baseline Statler.
• Score: 19.850681559877977
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in Vision-Language Models (VLMs) have enabled video-instructed robotic programming, allowing agents to interpret video demonstrations and generate executable control code. We formulate video-instructed robotic programming as a cross-domain adaptation problem, where perceptual and physical differences between demonstration and deployment induce procedural mismatches. However, current VLMs lack the procedural understanding needed to reformulate causal dependencies and achieve task-compatible behavior under such domain shifts. We introduce NeSyCR, a neurosymbolic counterfactual reasoning framework that enables verifiable adaptation of task procedures, providing a reliable synthesis of code policies. NeSyCR abstracts video demonstrations into symbolic trajectories that capture the underlying task procedure. Given deployment observations, it derives counterfactual states that reveal cross-domain incompatibilities. By exploring the symbolic state space with verifiable checks, NeSyCR proposes procedural revisions that restore compatibility with the demonstrated procedure. NeSyCR achieves a 31.14% improvement in task success over the strongest baseline Statler, showing robust cross-domain adaptation across both simulated and real-world manipulation tasks.

Related papers

• Towards Neuro-Symbolic Procedural Reasoning for Long-Horizon Vision-Language-Action Manipulation [12.461719086222788]
  Vision-language-action (VLA) models can execute short manipulation skills, but remain brittle in long-horizon procedures.<n>We investigate a neuro-symbolic framework that combines learned VLA control with explicit task graphs and multimodal procedural memory.
  arXiv  Detail & Related papers  (2026-09-04T17:14:50Z)
• FLARE: A Failure-Aware Framework for Autonomous Correction and Recovery in Visual-Language Robotic Manipulation [43.89036551517797]
  Vision-Language-Action Models (VLAs) have demonstrated significant promise in generalizing to complex, long-horizon robotic manipulation tasks.<n>Their performance remains brittle, as they are typically trained on trajectory-monotonic, failure-free demonstrations.<n>We propose FLARE, a novel framework that endows VLAs with robust error recovery capabilities through a Retry" and Reset" paradigm.
  arXiv  Detail & Related papers  (2026-08-27T05:58:06Z)
• PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution [49.776611937968]
  We present PhyAgentOS, a foundation delivering scheduling, verification, memory, benchmarking, and safety as system-level services.<n>Its Session-Centered treats a session, not an action, as the minimum unit of scheduling, compatibility, supervised execution, evidence collection, and acceptance.<n>SessionVerifier distinguishes execution termination from semantic task completion via evidence-grounded verdicts of success, failure, or replan.<n> Benchmarking reuses the deployment session and verification path, so results trace to real execution.
  arXiv  Detail & Related papers  (2026-07-18T04:46:53Z)
• Neural Procedural Memory: Empowering LLM Agents with Implicit Activation Steering [36.07257282655538]
  The paper introduces Neural Procedural Memory (NPM), a training-free framework that represents agent memory through implicit activation steering rather than explicit instructions.<n> Evaluations show that NPM performs comparably to baselines using explicit textual instructions.
  arXiv  Detail & Related papers  (2026-06-29T06:05:25Z)
• Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models [15.486973209713954]
  GTA-VLA(Guide, Think, Act) is an interactive Vision-Language-Action framework.<n>It enables spatially steerable embodied reasoning by allowing users to guide robot policies with explicit visual cues.
  arXiv  Detail & Related papers  (2026-05-13T14:58:29Z)
• ICLR: In-Context Imitation Learning with Visual Reasoning [7.622880558476133]
  In-context imitation learning enables robots to adapt to new tasks from a small number of demonstrations without additional training.<n>We present In-Context Imitation Learning with Visual Reasoning (ICLR), a novel framework that augments demonstration prompts with structured visual reasoning traces.
  arXiv  Detail & Related papers  (2026-03-08T08:40:05Z)
• \textsc{NaVIDA}: Vision-Language Navigation with Inverse Dynamics Augmentation [50.027425808733994]
  textscNaVIDA is a unified VLN framework that couples policy learning with action-grounded visual dynamics and adaptive execution.<n>textscNaVIDA augments training with chunk-based inverse-dynamics supervision to learn causal relationship between visual changes and corresponding actions.<n>Experiments show that textscNaVIDA achieves superior navigation performance compared to state-of-the-art methods with fewer parameters.
  arXiv  Detail & Related papers  (2026-01-26T06:16:17Z)
• From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuning [81.97788535387286]
  We propose a framework that internalizes the agentic verification-and-editing mechanism into a unified, single-pass inference process.<n>With minimal data, SRI-Coder enables Chat models to surpass the completion performance of their Base counterparts.<n>Unlike FIM-style tuning, SRI preserves general coding competencies and maintains inference latency comparable to standard FIM.
  arXiv  Detail & Related papers  (2026-01-19T20:33:53Z)
• Towards Reliable Code-as-Policies: A Neuro-Symbolic Framework for Embodied Task Planning [25.860785629018356]
  We propose a neuro-symbolic embodied task planning framework that incorporates explicit symbolic verification and interactive validation processes during code generation.<n>We evaluate our framework on RLBench and in real-world settings across dynamic, partially observable scenarios.
  arXiv  Detail & Related papers  (2025-10-24T10:01:08Z)
• EgoVIS@CVPR: What Changed and What Could Have Changed? State-Change Counterfactuals for Procedure-Aware Video Representation Learning [22.919192926764836]
  We study procedure-aware video representation learning by incorporating state-change descriptions.<n>We generate state-change counterfactuals that simulate hypothesized failure outcomes.<n>Our results demonstrate the effectiveness of the proposed state-change descriptions and their counterfactuals.
  arXiv  Detail & Related papers  (2025-05-30T13:39:29Z)
• Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy [73.75271615101754]
  We present Dita, a scalable framework that leverages Transformer architectures to directly denoise continuous action sequences.<n>Dita employs in-context conditioning -- enabling fine-grained alignment between denoised actions and raw visual tokens from historical observations.<n>Dita effectively integrates cross-embodiment datasets across diverse camera perspectives, observation scenes, tasks, and action spaces.
  arXiv  Detail & Related papers  (2025-03-25T15:19:56Z)
• Diffusion Transformer Policy [48.50988753948537]
  We propose a large multi-modal diffusion transformer, dubbed as Diffusion Transformer Policy, to model continuous end-effector actions.<n>By leveraging the scaling capability of transformers, the proposed approach can effectively model continuous end-effector actions across large diverse robot datasets.
  arXiv  Detail & Related papers  (2024-10-21T12:43:54Z)
• R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Models [50.19174067263255]
  We introduce prior preference learning techniques and self-revision schedules to help the agent excel in sparse-reward, continuous action, goal-based robotic control POMDP environments. We show that our agents offer improved performance over state-of-the-art models in terms of cumulative rewards, relative stability, and success rate.
  arXiv  Detail & Related papers  (2024-09-21T18:32:44Z)
• Trustworthy Image Semantic Communication with GenAI: Explainablity, Controllability, and Efficiency [59.15544887307901]
  Image semantic communication (ISC) has garnered significant attention for its potential to achieve high efficiency in visual content transmission. Existing ISC systems based on joint source-channel coding face challenges in interpretability, operability, and compatibility. We propose a novel trustworthy ISC framework that employs Generative Artificial Intelligence (GenAI) for multiple downstream inference tasks.
  arXiv  Detail & Related papers  (2024-08-07T14:32:36Z)
• Learning from Demonstrations using Signal Temporal Logic [1.2182193687133713]
  Learning-from-demonstrations is an emerging paradigm to obtain effective robot control policies. We use Signal Temporal Logic to evaluate and rank the quality of demonstrations. We show that our approach outperforms the state-of-the-art Maximum Causal Entropy Inverse Reinforcement Learning.
  arXiv  Detail & Related papers  (2021-02-15T18:28:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.