Multi-View Unified Camera Fields: Geometry-Shaped Action-Facing Representations for RGB-Only Multi-Camera VLA Policies

• URL: http://arxiv.org/abs/2608.01826v1
• Date: Mon, 03 Aug 2026 07:36:09 GMT
• Title: Multi-View Unified Camera Fields: Geometry-Shaped Action-Facing Representations for RGB-Only Multi-Camera VLA Policies
• Abstract summary: Multi-View Unified Camera Fields (MVUCF) is a training-only framework that forms a shared action-facing latent field across views.<n>After geometry injection, depth, camera calibration, and auxiliary heads are removed, so deployment uses the original RGB-only graph with no extra inference FLOPs.
• Score: 12.27577524307746
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action (VLA) models have shown strong generalization in robotic manipulation, yet complex contact-rich tasks often benefit from multi-camera observations that jointly capture the end effector, objects, and targets under occlusion. Existing multi-camera VLAs usually concatenate view tokens, leaving action representations weak in metric depth and inconsistent across cameras. We introduce Multi-View Unified Camera Fields (MVUCF), a training-only framework that forms a shared action-facing latent field across views. A coordinate-query depth objective makes metric depth recoverable, while a preprocessing-aware correspondence objective aligns tokens observing the same physical point from different cameras. Both directly shape the hidden states consumed by the action module. After geometry injection, depth, camera calibration, and auxiliary heads are removed, so deployment uses the original RGB-only graph with no extra inference FLOPs. Held-out probes confirm stronger depth recovery and cross-view matching. Under matched GR00T-N1.6 settings, MVUCF reaches 98.9% on LIBERO, improves LIBERO-Plus by 22.4 points, and raises success by 23.3 points across six RoboTwin tasks spanning three action families: touch, move-and-place, and contact interaction. Real-world humanoid experiments further provide evidence of its practical effectiveness under RGB-only deployment.

Related papers

• Spatially Conditioned Diffusion Policy: Learning Precise and Robust Manipulation with a Single RGB Camera [6.648702147742411]
  Spatially Conditioned Diffusion Policy (SCDP) is a diffusion-based visuomotor policy that achieves precise and robust manipulation in a single-camera setting.<n>SCDP consists of two key components: (i) a visual encoder that produces multi-scale feature maps to capture both broader context and fine-grained visual features, and (ii) a spatial conditioning module that samples point-wise features along intermediate end-effector trajectories in the diffusion loop.
  arXiv  Detail & Related papers  (2026-06-12T15:12:03Z)
• Object Pose Transformer: Unifying Unseen Object Pose Estimation [54.20344997573707]
  Learning model-free object pose estimation for unseen instances remains a fundamental challenge in 3D vision.<n>ours jointly predicts depth, point maps, camera parameters, and normalized object coordinates from RGB inputs.<n>ours is camera-agnostic, learning camera intrinsics on-the-fly and supporting optional depth input for metric-scale recovery.
  arXiv  Detail & Related papers  (2026-03-24T16:04:16Z)
• Pixel2Catch: Multi-Agent Sim-to-Real Transfer for Agile Manipulation with a Single RGB Camera [3.219880761967806]
  This work focuses on a novel approach that recognizes object motion using pixel-level visual information extracted from a single RGB image.<n>To achieve stable learning in a high-DoF system composed of a robot arm equipped with a multi-fingered hand, we design a heterogeneous multi-agent reinforcement learning framework.<n>Each agent is trained cooperatively using role-specific observations and rewards, and the learned policies are successfully transferred from simulation to the real world.
  arXiv  Detail & Related papers  (2026-02-26T08:15:38Z)
• DVPE: Divided View Position Embedding for Multi-View 3D Object Detection [7.791229698270439]
  Current research faces challenges in balancing between receptive fields and reducing interference when aggregating multi-view features. This paper proposes a divided view method, in which features are modeled globally via the visibility crossattention mechanism, but interact only with partial features in a divided local virtual space. Our framework, named DVPE, achieves state-of-the-art performance (57.2% mAP and 64.5% NDS) on the nuScenes test set.
  arXiv  Detail & Related papers  (2024-07-24T02:44:41Z)
• DVANet: Disentangling View and Action Features for Multi-View Action Recognition [56.283944756315066]
  We present a novel approach to multi-view action recognition where we guide learned action representations to be separated from view-relevant information in a video. Our model and method of training significantly outperforms all other uni-modal models on four multi-view action recognition datasets.
  arXiv  Detail & Related papers  (2023-12-10T01:19:48Z)
• Geometric-aware Pretraining for Vision-centric 3D Object Detection [77.7979088689944]
  We propose a novel geometric-aware pretraining framework called GAPretrain. GAPretrain serves as a plug-and-play solution that can be flexibly applied to multiple state-of-the-art detectors. We achieve 46.2 mAP and 55.5 NDS on the nuScenes val set using the BEVFormer method, with a gain of 2.7 and 2.1 points, respectively.
  arXiv  Detail & Related papers  (2023-04-06T14:33:05Z)
• Coordinate-Aligned Multi-Camera Collaboration for Active Multi-Object Tracking [114.16306938870055]
  We propose a coordinate-aligned multi-camera collaboration system for AMOT. In our approach, we regard each camera as an agent and address AMOT with a multi-agent reinforcement learning solution. Our system achieves a coverage of 71.88%, outperforming the baseline method by 8.9%.
  arXiv  Detail & Related papers  (2022-02-22T13:28:40Z)
• RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video [76.86512780916827]
  We present the first real-time method for motion capture of skeletal pose and 3D surface geometry of hands from a single RGB camera. In order to address the inherent depth ambiguities in RGB data, we propose a novel multi-task CNN. We experimentally verify the individual components of our RGB two-hand tracking and 3D reconstruction pipeline.
  arXiv  Detail & Related papers  (2021-06-22T12:53:56Z)
• Self-supervised Human Detection and Segmentation via Multi-view Consensus [116.92405645348185]
  We propose a multi-camera framework in which geometric constraints are embedded in the form of multi-view consistency during training. We show that our approach outperforms state-of-the-art self-supervised person detection and segmentation techniques on images that visually depart from those of standard benchmarks.
  arXiv  Detail & Related papers  (2020-12-09T15:47:21Z)
• MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion [19.034317851914725]
  We present a system which can estimate the accurate poses of multiple known objects in contact and occlusion from real-time, embodied multi-view vision. Our approach makes 3D object pose proposals from single RGB-D views, accumulates pose estimates and non-parametric occupancy information from multiple views as the camera moves. We verify the accuracy and robustness of our approach experimentally on 2 object datasets: YCB-Video, and our own challenging Cluttered YCB-Video.
  arXiv  Detail & Related papers  (2020-04-09T02:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.