Geometric Action Model for Robot Policy Learning
Abstract Overview
This paper proposes the Geometric Action Model (GAM), a language-conditioned robot manipulation policy that repurposes a pretrained geometric foundation model as a shared backbone for perception, temporal prediction, and action decoding. GAM splits the geometric model at an intermediate layer, uses the shallow layers to encode observations, inserts a causal predictor that forecasts future latent geometry conditioned on language, proprioception, and action history, and then routes the predicted tokens through the remaining backbone layers to decode both future geometry and actions. The method is designed to make 3D geometric reasoning explicit inside the policy rather than relying mainly on 2D image priors. Experiments cover simulation benchmarks such as LIBERO and LIBERO-Plus, additional RoboCasa-Kitchen results, and real-robot manipulation tasks. Across these settings, the evidence indicates improved robustness to perturbations together with lower latency and smaller model size than several foundation-model-scale baselines.
Novelty
The distinctive contribution is to turn a pretrained geometric foundation model into a unified world-action policy by inserting a causal temporal predictor inside the backbone, rather than using geometry only as a frozen feature extractor or auxiliary alignment target. The model jointly predicts future geometric tokens and action tokens in a shared latent space and reuses the deep geometric decoder blocks to refine both predictions.
Results
On LIBERO-Plus, GAM achieves an 85.5% overall success rate versus 97.6% on original LIBERO, corresponding to a 12.1-point drop, and it reports especially strong camera-perturbation performance at 83.1%, a 9.7-point gain over the strongest listed baseline in that column. The paper also reports 6.9 ms inference latency, about 145 Hz control, and up to 55× speedup over the diffusion-based Cosmos-Policy baseline while using a 1.4B-parameter model. In real-world experiments, the authors report that GAM outperforms the compared baselines and remains more robust under external camera perturbations.
Key Points
- GAM integrates perception, future geometric prediction, and action decoding within one shared geometric foundation model by splitting the backbone and inserting a causal future predictor.
- The empirical evidence emphasizes robustness under distribution shift, particularly camera perturbations, where explicit 3D geometric priors appear to improve generalization over 2D-centric VLA and WAM baselines.
- The method is positioned as both efficient and compact, with substantially lower inference latency and fewer parameters than several large baseline policies while maintaining strong benchmark performance.