ICPRL: Acquiring Physical Intuition from Interactive Control

• URL: http://arxiv.org/abs/2603.13295v1
• Date: Sun, 01 Mar 2026 15:17:18 GMT
• Title: ICPRL: Acquiring Physical Intuition from Interactive Control
• Abstract summary: We introduce ICPRL (In-Context Physical Reinforcement Learning), a framework that empowers VLMs to acquire physical intuition and adapt their policies in-context.<n>Our approach trains a vision-grounded policy model via multi-turn Group Relative Policy Optimization (GRPO) over diverse multi-episode interaction histories.<n>This enables the agent to adapt strategies by conditioning on past trial-and-error sequences, without requiring any weight updates.
• Score: 38.098959182766144
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: VLMs excel at static perception but falter in interactive reasoning in dynamic physical environments, which demands planning and adaptation to dynamic outcomes. Existing physical reasoning methods often depend on abstract symbolic inputs or lack the ability to learn and adapt from direct, pixel-based visual interaction in novel scenarios. We introduce ICPRL (In-Context Physical Reinforcement Learning), a framework inspired by In-Context Reinforcement Learning (ICRL) that empowers VLMs to acquire physical intuition and adapt their policies in-context. Our approach trains a vision-grounded policy model via multi-turn Group Relative Policy Optimization (GRPO) over diverse multi-episode interaction histories. This enables the agent to adapt strategies by conditioning on past trial-and-error sequences, without requiring any weight updates. This adaptive policy works in concert with a separately trained world model that provides explicit physical reasoning by predicting the results of potential actions. At inference, the policy proposes candidate actions, while the world model predicts outcomes to guide a root-node PUCT search to select the most promising action. Evaluated on the diverse physics-based puzzle-solving tasks in the DeepPHY benchmark, ICPRL demonstrates significant improvements across both its I. policy-only, and II. world-model-augmented stages. Notably, these gains are retained in unseen physical environments, demonstrating that our framework facilitates genuine in-context acquisition of the environment's physical dynamics from interactive experience.

Related papers

• EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learning [57.312638079204255]
  Training large language model agents for long-horizon tool use typically relies on interactions with real or synthesized executable environments.<n>We introduce EnvACE, an agentic reinforcement learning method that replaces external environment interaction during training with world rehearsal.
  arXiv  Detail & Related papers  (2026-08-06T15:54:36Z)
• What Matters in Humanoid General Motion Tracking? An Empirical Study [2.728639503114606]
  We develop YAHMP, an open-source framework for training, evaluating, and deploying whole-body motion tracking policies on the Unitree G1.<n>We evaluate the resulting policies on a test set of retargeted human motions and compare the nominal policy with TWIST2 as an external baseline trained on the same motion set.
  arXiv  Detail & Related papers  (2026-07-22T08:37:19Z)
• Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimization [56.150184676745205]
  Video anomaly understanding (VAU) relies on sparse, context-dependent cues.<n>We propose $Anomtext-$, a closed-loop framework that reconceptualizes video understanding as an active sequential decision-making process.<n>Inspired by human video-reviewing behavior, this framework unifies internal cognitive reasoning and strategic evidence acquisition into an interleaved policy.
  arXiv  Detail & Related papers  (2026-07-01T08:41:49Z)
• AdaWorldPolicy: World-Model-Driven Diffusion Policy with Online Adaptive Learning for Robotic Manipulation [12.592383721590402]
  We introduce a unified framework, World-Model-Driven Diffusion Policy with Online Adaptive Learning (AdaWorldPolicy)<n>Our core insight is that world models provide strong supervision signals, enabling online adaptive learning in dynamic environments.<n>AdaWorldPolicy achieves state-of-the-art performance, with dynamical adaptive capacity to out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2026-02-23T17:12:25Z)
• PolicyEvol-Agent: Evolving Policy via Environment Perception and Self-Awareness with Theory of Mind [9.587070290189507]
  PolicyEvol-Agent is a comprehensive framework characterized by systematically acquiring intentions of others.<n>PolicyEvol-Agent integrates a range of cognitive operations with Theory of Mind alongside internal and external perspectives.
  arXiv  Detail & Related papers  (2025-04-20T06:43:23Z)
• Efficient Policy Adaptation with Contrastive Prompt Ensemble for Embodied Agents [6.402396836189286]
  We present a novel contrastive prompt ensemble (ConPE) framework for embodied reinforcement learning.<n>We devise a guided-attention-based ensemble approach with multiple visual prompts on the vision-language model to construct robust state representations.<n>In experiments, we show that ConPE outperforms other state-of-the-art algorithms for several embodied agent tasks.
  arXiv  Detail & Related papers  (2024-12-16T06:53:00Z)
• Free Energy Projective Simulation (FEPS): Active inference with interpretability [40.11095094521714]
  Free Energy Projective Simulation (FEP) and active inference (AIF) have achieved many successes. Recent work has focused on improving such agents' performance in complex environments by incorporating the latest machine learning techniques. We introduce Free Energy Projective Simulation (FEPS) to model agents in an interpretable way without deep neural networks.
  arXiv  Detail & Related papers  (2024-11-22T15:01:44Z)
• Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning [51.52387511006586]
  We propose Hierarchical Opponent modeling and Planning (HOP), a novel multi-agent decision-making algorithm. HOP is hierarchically composed of two modules: an opponent modeling module that infers others' goals and learns corresponding goal-conditioned policies. HOP exhibits superior few-shot adaptation capabilities when interacting with various unseen agents, and excels in self-play scenarios.
  arXiv  Detail & Related papers  (2024-06-12T08:48:06Z)
• OMPO: A Unified Framework for RL under Policy and Dynamics Shifts [42.57662196581823]
  Training reinforcement learning policies using environment interaction data collected from varying policies or dynamics presents a fundamental challenge. Existing works often overlook the distribution discrepancies induced by policy or dynamics shifts, or rely on specialized algorithms with task priors. In this paper, we identify a unified strategy for online RL policy learning under diverse settings of policy and dynamics shifts: transition occupancy matching.
  arXiv  Detail & Related papers  (2024-05-29T13:36:36Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)
• Invariant Causal Imitation Learning for Generalizable Policies [87.51882102248395]
  We propose Invariant Causal Learning (ICIL) to learn an imitation policy. ICIL learns a representation of causal features that is disentangled from the specific representations of noise variables. We show that ICIL is effective in learning imitation policies capable of generalizing to unseen environments.
  arXiv  Detail & Related papers  (2023-11-02T16:52:36Z)
• A Regularized Implicit Policy for Offline Reinforcement Learning [54.7427227775581]
  offline reinforcement learning enables learning from a fixed dataset, without further interactions with the environment. We propose a framework that supports learning a flexible yet well-regularized fully-implicit policy. Experiments and ablation study on the D4RL dataset validate our framework and the effectiveness of our algorithmic designs.
  arXiv  Detail & Related papers  (2022-02-19T20:22:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.