PhysCaP: Grounding Code-as-Policy Agent with Physics-Informed Exploration
Abstract Overview
PhysCaP is a physics-informed code-as-policy agent designed for active perception in robotic manipulation tasks where success depends on latent physical properties that cannot be inferred from vision alone. The system extends code-as-policy with training-free physical property extraction modules that estimate object mass and stiffness from robot proprioception, without requiring additional sensing hardware. It also uses a dual-agent exploration design: a Planner decides when exploration is needed and when to stop, while a Prioritizer filters and ranks candidate interactions to reduce unnecessary probing. The paper evaluates the approach on three real-world tabletop tasks and a simulated LIBERO task involving hidden-state estimation such as identifying an empty can or a ripe avocado.
Novelty
The main novelty is the integration of explicit physics-informed exploration into a code-as-policy framework, allowing the agent to seek missing physical information through interaction rather than relying on passive observation. The work is also distinctive in combining training-free proprioceptive mass and stiffness estimation with a separate Planner and Prioritizer to structure efficient exploration.
Results
Across real-world tasks, PhysCaP generally matched or exceeded PhysX-enabled baselines while using fewer object interactions and less execution time, achieving the best overall balance of success, efficiency, and runtime among compared methods. In simulation on the LIBERO empty-container task, it reached the highest reported success rate (78%) with 1.44 average interactions, outperforming evaluated VLA baselines that largely failed under partial observability. Ablations further support the accuracy of the physical property modules and show that separating planning from prioritization outperforms merging them into a single agent.
Key Points
- PhysCaP augments code-as-policy with active, physics-informed exploration for tasks requiring hidden physical-state inference.
- The method introduces training-free proprioceptive modules for estimating object mass and stiffness using a standard gripper and robot feedback only.
- Experiments in real-world tabletop tasks and LIBERO simulation show improved interaction efficiency and stronger robustness than passive or naive interactive baselines.