IMPACT-VLA: Interaction-aware Multimodal Propagation Attribution via Counterfactual Trajectories for Vision-Language-Action Policies

• URL: http://arxiv.org/abs/2609.15005v1
• Date: Mon, 14 Sep 2026 04:13:17 GMT
• Title: IMPACT-VLA: Interaction-aware Multimodal Propagation Attribution via Counterfactual Trajectories for Vision-Language-Action Policies
• Abstract summary: Vision-Language-Action Policies (VLA) perform robot manipulation tasks using multimodal inputs.<n>It remains unclear at which execution stages each modality contributes to final task success.<n>We propose Interaction-aware Multimodal Propagation Attribution via Counterfactual Trajectories for Vision-Language-Action Policies.
• Score: 3.5795275871379704
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action (VLA) policies perform robot manipulation tasks using multimodal inputs such as visual observations, proprioceptive states, and language instructions. However, it remains unclear at which execution stages each modality contributes to final task success and how input interventions propagate through subsequent states, observations, and actions. Existing attribution approaches primarily measure local sensitivity or temporally aggregated importance, limiting their ability to capture phase-dependent contributions and cross-phase dependencies. We propose Interaction-aware Multimodal Propagation Attribution via Counterfactual Trajectories for Vision-Language-Action Policies (IMPACT-VLA). IMPACT-VLA constructs behavioral phases from action transitions in a successful reference rollout, aligns them with policy query boundaries, and defines phase-modality blocks as attribution units. It then performs closed-loop counterfactual re-execution to quantify each block's contribution to final task success. We further analyze cross-phase non-additive interactions and trajectory propagation while distinguishing behavioral from functional recovery. Across 30 LIBERO robot manipulation tasks using OpenVLA-OFT, dominant-modality transitions occurred in 25 tasks (83.3%), and closed-loop attribution identified task-critical information more faithfully than Static Action Perturbation. Later-block marginal gains for negatively interacting pairs increased by approximately 3.3x under early-phase input replacement, while functional recovery could occur without behavioral recovery. These results reveal when multimodal inputs support task success and how their contributions become conditionally coupled during closed-loop execution.

Related papers

• UniMPA: A Unified Memory-Prediction-Action Model via Action-Grounded Transition Modeling [73.35395707747456]
  Recent advances in Vision-Language-Action models have improved robotic manipulation, yet observation-to-action learning remains limited by a fundamental transition realizability gap.<n>A visually plausible predicted future observation does not necessarily correspond to a physically realizable transition.<n>A historically executable action pattern may not necessarily realize the intended transition in the current scene and therefore requires context-aware adaptation.
  arXiv  Detail & Related papers  (2026-09-10T17:45:03Z)
• VLA-Corrector: Stage-Aware Observable State Understanding for Prompt-Based Closed-Loop Recovery of Vision-Language-Action Policies [12.698552476812855]
  Long-horizon robot manipulation with Vision-Language-Action (VLA) policies remains vulnerable to execution-time deviations.<n>We introduce a stage-aware failure verification and Prompt Recovery framework that enables closed-loop correction of a fixed VLA policy.
  arXiv  Detail & Related papers  (2026-09-06T09:50:30Z)
• PACE: Phase-Progress-Aware Credit for Long-Horizon Embodied Manipulation [17.66354304197193]
  Policy improvement requires step-level credit signals to distinguish behaviors that advance the task from those that stall or regress.<n>We present PACE, a credit-assignment framework for post-training on long-horizon manipulation.
  arXiv  Detail & Related papers  (2026-08-15T04:25:16Z)
• Temporal GRPO: Beyond Trajectory-Level Credit in Vision-Language-Action Reinforcement Learning [27.507258147328]
  Outcome-driven reinforcement learning offers a scalable way to post-train vision-language-action (VLA) policies from sparse task-success feedback.<n>On RoboTwin 2.0, Temporal GRPO improves task success and sample efficiency, with consistent gains across task horizons.
  arXiv  Detail & Related papers  (2026-08-13T09:54:14Z)
• AutoIntervene: Calibrated Intervention for Action-Chunking Imitation Learning Policies [4.786936073180585]
  Action-chunking visuomotor policies learn from demonstrations and improve temporal consistency.<n>We present AutoIntervene, an online framework that selectively transfers control between an action-chunking policy and an operator during deployment.
  arXiv  Detail & Related papers  (2026-08-07T10:14:29Z)
• SegDiff: Segmented Trajectory Diffusion for Consistent and Adaptive Robot Manipulation [65.81476183180527]
  We introduce SegDiff, a closed-loop visuomotor policy that integrates the strengths of both paradigms.<n>SegDiff decomposes demonstrations into motion segments between keyposes and learns to predict the continuous trajectory from the current state to the next keypose.<n>SegDiff demonstrates significant performance gains over existing approaches across various simulated and real-world scenarios.
  arXiv  Detail & Related papers  (2026-07-13T02:48:03Z)
• PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulation [27.791908160098625]
  PALM builds policy learning around interaction-centric affordance reasoning and subtask progress cues.<n>Palm consistently outperforms baselines in simulation and real-world experiments.
  arXiv  Detail & Related papers  (2026-01-11T21:00:58Z)
• Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach [78.4812458793128]
  We propose textbfTACO, a test-time-scaling framework that applies a lightweight pseudo-count estimator as a high-fidelity verifier of action chunks.<n>Our method resembles the classical anti-exploration principle in offline reinforcement learning (RL), and being gradient-free, it incurs significant computational benefits.
  arXiv  Detail & Related papers  (2025-12-02T14:42:54Z)
• ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Context [54.58057019521198]
  Leveraging temporal context is crucial for success in partially observable robotic tasks.<n>Prior work in behavior cloning has demonstrated inconsistent performance gains when using multi-frame observations.<n>We introduce ContextVLA, a policy model that robustly improves robotic task performance by effectively leveraging multi-frame observations.
  arXiv  Detail & Related papers  (2025-10-05T15:29:57Z)
• PARL-MT: Learning to Call Functions in Multi-Turn Conversation with Progress Awareness [57.020401590532686]
  We introduce PARL-MT, a framework that explicitly incorporates progress awareness into LLM training for multi-turn function calling.<n>We show that PARL-MT significantly outperforms existing methods, highlighting the effectiveness of progress awareness in enabling robust and efficient multi-turn function calling.
  arXiv  Detail & Related papers  (2025-09-27T09:32:27Z)
• Explaining multimodal LLMs via intra-modal token interactions [55.27436637894534]
  Multimodal Large Language Models (MLLMs) have achieved remarkable success across diverse vision-language tasks, yet their internal decision-making mechanisms remain insufficiently understood.<n>We propose enhancing interpretability by leveraging intra-modal interaction.
  arXiv  Detail & Related papers  (2025-09-26T14:39:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.