Closing the Loop in Humanoid VLA: Persistent 3D Object Tokens for Verifiable Loco-Manipulation

• URL: http://arxiv.org/abs/2607.18016v2
• Date: Sun, 26 Jul 2026 09:19:57 GMT
• Title: Closing the Loop in Humanoid VLA: Persistent 3D Object Tokens for Verifiable Loco-Manipulation
• Abstract summary: Long-horizon humanoid loco-manipulation requires the robot to treat task objects as persistent physical entities across movement, contact, and recovery.<n>We propose emphPersistent Object Tokenization (POT), which maintains role-indexed 3D object records from RGB-D observations.<n>Instantiated as emphPOT-VLA, the same object records condition action generation and support geometric predicate checks, yielding a closed-loop execution system.
• Score: 14.431337587744538
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language-action policies are a promising foundation for general robot control, but long-horizon humanoid loco-manipulation requires the robot to treat task objects as persistent physical entities across movement, contact, occlusion, and recovery. We study this problem as object-state divergence: the object state used to condition a whole-body action can differ from the state used to decide whether the action achieved the intended physical relation. We propose \emph{Persistent Object Tokenization} (POT), which maintains role-indexed 3D object records from RGB-D observations and converts them into object tokens for a whole-body action expert. Instantiated as \emph{POT-VLA}, the same object records condition action generation and support geometric predicate checks, yielding a closed-loop execution system in which object state is both actionable and verifiable. On a Unitree G1, POT-VLA improves a matched direct GR00T-N1.7 baseline from 39/80 to 71/80 successes over eight real-world task families. In an external Being-0-aligned reference, POT-VLA achieves 44/50 successes on aligned service tasks, compared with the 37/50 success reported by the Being-0 paper. The largest gains occur on tasks requiring maintained 3D relations, suggesting that persistent object-centered state is a useful abstraction for verifiable humanoid VLA execution.

Related papers

• AffordTrajDP: Dynamic Affordance-Guided Visuomotor Policy Learning for Robotic Manipulation [8.422659700538636]
  AffordTrajDP constructs affordance trajectories via object-centric temporal propagation to guide the progressive manipulation process.<n>AffordTrajDP achieves 70.0% average success rate on ManiSkill3, outperforming strong baselines by up to 17.8%.
  arXiv  Detail & Related papers  (2026-08-03T02:19:37Z)
• Sparse2Act: Learning Action-Aligned Sparse 3D Representations for Cross-Domain Robot Manipulation [52.00134393320209]
  We introduce Sparse2Act, an observation-action alignment framework for pretraining sparse point-cloud encoders.<n>Masked sparse 3D tokens are trained to organize scene features around the workspace motion paired with the observation.<n>On the LIBERO-10 benchmark, our method achieves 86.9% average success after 500 fine-tuning steps.
  arXiv  Detail & Related papers  (2026-06-10T23:56:01Z)
• GEAR-VLA: Learning Geometry-Aware Action Representations for Generalizable Robotic Manipulation [49.16739604572808]
  Vision-Language-Action (VLA) models achieve strong benchmark performance but struggle in real-world deployment with unseen objects.<n>We argue that this stems from the lack of a unified geometry-aware manipulation representation.<n>We propose GEAR-VLA, a VLA framework for learning unified geometry-aware action representations for generalizable robotic manipulation.
  arXiv  Detail & Related papers  (2026-06-07T09:23:16Z)
• Imagine2Real: Towards Zero-shot Humanoid-Object Interaction via Video Generative Priors [51.096845970243855]
  Whole-body Humanoid-Object Interaction (HOI) is bottlenecked by the scarcity of high-fidelity 3D data.<n>We propose Imagine2Real, a zero-shot HOI framework for flexible, geometry-free interaction.
  arXiv  Detail & Related papers  (2026-05-21T10:15:39Z)
• VOFA: Visual Object Goal Pushing with Force-Adaptive Control for Humanoids [35.671376073584305]
  VOFA is a visual goal-conditioned humanoid loco-manipulation system.<n>It is capable of pushing objects with unknown physical properties to arbitrary goal positions.<n>VOFA is extensively evaluated in both simulation and real-world experiments on the Booster T1 humanoid robot.
  arXiv  Detail & Related papers  (2026-05-02T16:16:23Z)
• Cybo-Waiter: A Physical Agentic Framework for Humanoid Whole-Body Locomotion-Manipulation [14.648223907591849]
  Long horizon execution is challenging for humanoids because locomotion and manipulation are tightly coupled through stance, reachability, and balance.<n>We present a humanoid agent framework that turns VLM plans into verifiable task programs and closes the loop with multi object 3D supervision.<n>Experiments on tabletop manipulation and long horizon humanoid loco manipulation tasks show improved robustness from multi object grounding, temporal stability, and recovery driven replanning.
  arXiv  Detail & Related papers  (2026-03-11T11:41:32Z)
• Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation [95.89924101984566]
  We introduce OptimusVLA, a dual-memory VLA framework with Global Prior Memory (GPM) and Local Consistency Memory (LCM)<n>GPM replaces Gaussian noise with task-level priors retrieved from semantically similar trajectories.<n>LCM injects a learned consistency constraint that enforces temporal coherence and smoothness of trajectory.
  arXiv  Detail & Related papers  (2026-02-22T15:39:34Z)
• Articulated 3D Scene Graphs for Open-World Mobile Manipulation [55.97942733699124]
  We present MoMa-SG, a framework for building semantic-kinematic 3D scene graphs of articulated scenes.<n>We estimate articulation models using a novel unified twist estimation formulation.<n>We also introduce the novel Arti4D-Semantic dataset.
  arXiv  Detail & Related papers  (2026-02-18T10:40:35Z)
• Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge [62.981429762309226]
  The ability to actively ground task instructions from an egocentric view is crucial for AI agents to accomplish tasks or assist humans virtually. We propose to improve phrase grounding models' ability on localizing the active objects by: learning the role of objects undergoing change and extracting them accurately from the instructions. We evaluate our framework on Ego4D and Epic-Kitchens datasets.
  arXiv  Detail & Related papers  (2023-10-23T16:14:05Z)
• HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulation [29.01984677695523]
  We introduce Hybrid Actor-Critic Maps for Manipulation (HACMan), a reinforcement learning approach for 6D non-prehensile manipulation of objects. We evaluate HACMan on a 6D object pose alignment task in both simulation and in the real world. Compared to alternative action representations, HACMan achieves a success rate more than three times higher than the best baseline.
  arXiv  Detail & Related papers  (2023-05-06T05:55:27Z)
• SORNet: Spatial Object-Centric Representations for Sequential Manipulation [39.88239245446054]
  Sequential manipulation tasks require a robot to perceive the state of an environment and plan a sequence of actions leading to a desired goal state. We propose SORNet, which extracts object-centric representations from RGB images conditioned on canonical views of the objects of interest.
  arXiv  Detail & Related papers  (2021-09-08T19:36:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.