ICLR: In-Context Imitation Learning with Visual Reasoning

• URL: http://arxiv.org/abs/2603.07530v1
• Date: Sun, 08 Mar 2026 08:40:05 GMT
• Title: ICLR: In-Context Imitation Learning with Visual Reasoning
• Abstract summary: In-context imitation learning enables robots to adapt to new tasks from a small number of demonstrations without additional training.<n>We present In-Context Imitation Learning with Visual Reasoning (ICLR), a novel framework that augments demonstration prompts with structured visual reasoning traces.
• Score: 7.622880558476133
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In-context imitation learning enables robots to adapt to new tasks from a small number of demonstrations without additional training. However, existing approaches typically condition only on state-action trajectories and lack explicit representations of task intent. This limitation hinders performance in complex and ambiguous task settings where the same actions may be consistent with different objectives. To address this, we present In-Context Imitation Learning with Visual Reasoning (ICLR), a novel framework that augments demonstration prompts with structured visual reasoning traces representing anticipated future robot trajectories in image space. ICLR also jointly learns to generate reasoning traces and low-level actions within a unified autoregressive transformer, enabling the model to mimic not only action prediction but also the reasoning process that leads to those actions. We extensively evaluate ICLR in both simulation and real-world manipulation tasks and demonstrate consistent improvements in success rates and generalization to unseen tasks and novel object configurations compared to other in-context imitation learning methods. These results suggest that incorporating embodied visual reasoning represents a promising direction for enhancing the robustness and generalization of robotic in-context learning systems.

Related papers

• Artificial Foveated Perception for Mitigating Shortcut Learning in Robotic Foundation Models [28.23835152537988]
  We propose Artificial Foveated Perception (AFP), a lightweight, policy-agnostic module for robotic foundation models.<n>AFP predicts task-conditioned masks over relevant objects, the robot, and other action-critical regions.<n>We show AFP reduces fine-tuning time, suppresses overfitting, and improves generalization under environmental perturbations.
  arXiv  Detail & Related papers  (2026-07-12T08:42:36Z)
• PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations [66.94988600664574]
  We present textbfPRTS (textbfPrimitive textbfReasoning and textbfTasking textbfSystem), a VLA foundation model that reformulates pretraining through Goal-Conditioned Reinforcement Learning.
  arXiv  Detail & Related papers  (2026-04-30T06:14:02Z)
• Behavior-Aware Item Modeling via Dynamic Procedural Solution Representations for Knowledge Tracing [29.193697409059613]
  We propose Behavior-Aware Item Modeling (BAIM), a framework that enriches item representations by integrating dynamic procedural solution information.<n>BAIM leverages a reasoning language model to decompose each item's solution into four problem-solving stages.<n> Experiments on XES3G5M and NIPS34 show that BAIM consistently outperforms strong pretraining-based baselines.
  arXiv  Detail & Related papers  (2026-04-09T13:47:39Z)
• Closed-Loop Verbal Reinforcement Learning for Task-Level Robotic Planning [1.6641413441634143]
  We propose a new Verbal Reinforcement Learning framework for interpretable task-level planning in mobile robotic systems.<n>The framework follows a closed-loop architecture that enables iterative policy improvement through interaction with the physical environment.<n>We validate the proposed framework on a real mobile robot performing a multi-stage manipulation and navigation task under execution uncertainty.
  arXiv  Detail & Related papers  (2026-03-23T16:28:36Z)
• Language-Grounded Decoupled Action Representation for Robotic Manipulation [78.42228162226839]
  We propose the Language-Grounded Decoupled Action Representation (LaDA) framework to connect perception and control.<n>LaDA introduces a fine-grained intermediate layer of three interpretable action primitives--translation, rotation, and gripper control--providing explicit semantic structure for low-level actions.<n>It further employs a semantic-guided soft-label contrastive learning objective to align similar action primitives across tasks, enhancing generalization and motion consistency.
  arXiv  Detail & Related papers  (2026-03-13T13:08:26Z)
• AR-VLA: True Autoregressive Action Expert for Vision-Language-Action Models [36.00004339916959]
  We propose a standalone autoregressive (AR) Action Expert that generates actions as a continuous causal sequence.<n>Our work introduces a scalable, context-aware action generation schema that provides a robust structural foundation for training effective robotic policies.
  arXiv  Detail & Related papers  (2026-03-10T18:03:29Z)
• \textsc{NaVIDA}: Vision-Language Navigation with Inverse Dynamics Augmentation [50.027425808733994]
  textscNaVIDA is a unified VLN framework that couples policy learning with action-grounded visual dynamics and adaptive execution.<n>textscNaVIDA augments training with chunk-based inverse-dynamics supervision to learn causal relationship between visual changes and corresponding actions.<n>Experiments show that textscNaVIDA achieves superior navigation performance compared to state-of-the-art methods with fewer parameters.
  arXiv  Detail & Related papers  (2026-01-26T06:16:17Z)
• Do-Undo: Generating and Reversing Physical Actions in Vision-Language Models [57.71440995598757]
  We introduce the Do-Undo task and benchmark to address a critical gap in vision-language models.<n>Do-Undo requires models to simulate the outcome of a physical action and then accurately reverse it, reflecting true cause-and-effect in the visual world.
  arXiv  Detail & Related papers  (2025-12-15T18:03:42Z)
• Learning Affordances at Inference-Time for Vision-Language-Action Models [50.93181349331096]
  In robotics, Vision-Language-Action models (VLAs) offer a promising path towards solving complex control tasks.<n>We introduce Learning from Inference-Time Execution (LITEN), which connects a VLA low-level policy to a high-level VLM that conditions on past experiences.<n>Our approach iterates between a reasoning phase that generates and executes plans for the low-level VLA, and an assessment phase that reflects on the resulting execution.
  arXiv  Detail & Related papers  (2025-10-22T16:43:29Z)
• Executable Analytic Concepts as the Missing Link Between VLM Insight and Precise Manipulation [70.8381970762877]
  Vision-Language Models (VLMs) have demonstrated remarkable capabilities in semantic reasoning and task planning.<n>We introduce GRACE, a novel framework that grounds VLM-based reasoning through executable analytic concepts.<n>G GRACE provides a unified and interpretable interface between high-level instruction understanding and low-level robot control.
  arXiv  Detail & Related papers  (2025-10-09T09:08:33Z)
• What to Do Next? Memorizing skills from Egocentric Instructional Video [43.59787683244105]
  We present a novel task, interactive action planning, and propose an approach that combines topological affordance memory with transformer architecture.<n>Our experimental results demonstrate that the proposed approach learns meaningful representations, resulting in improved performance and robust when action deviations occur.
  arXiv  Detail & Related papers  (2025-07-01T22:53:41Z)
• Visual In-Context Learning for Large Vision-Language Models [62.5507897575317]
  In Large Visual Language Models (LVLMs) the efficacy of In-Context Learning (ICL) remains limited by challenges in cross-modal interactions and representation disparities. We introduce a novel Visual In-Context Learning (VICL) method comprising Visual Demonstration Retrieval, Intent-Oriented Image Summarization, and Intent-Oriented Demonstration Composition. Our approach retrieves images via ''Retrieval & Rerank'' paradigm, summarises images with task intent and task-specific visual parsing, and composes language-based demonstrations.
  arXiv  Detail & Related papers  (2024-02-18T12:43:38Z)
• Sample-Efficient Reinforcement Learning with Symmetry-Guided Demonstrations for Robotic Manipulation [7.099237102357281]
  Reinforcement learning (RL) suffers from low sample efficiency, particularly in high-dimensional continuous state-action spaces.<n>We introduce Demo-EASE, a novel training framework using a dual-buffer architecture that stores both demonstrations and RL-generated experiences.<n>Our results show that Demo-EASE significantly accelerates convergence and improves final performance compared to standard RL baselines.
  arXiv  Detail & Related papers  (2023-04-12T11:38:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.