Selective Cross-View Consistency for World Action Models: Held-Out Viewpoint Robustness Without Test-Time Camera Information

• URL: http://arxiv.org/abs/2608.21402v1
• Date: Fri, 07 Aug 2026 12:03:23 GMT
• Title: Selective Cross-View Consistency for World Action Models: Held-Out Viewpoint Robustness Without Test-Time Camera Information
• Abstract summary: World action models (WAMs) jointly denoise future video frames and robot actions, and the video prior is expected to generalize their control.<n>We study a question specific to this model class: when training with same-state cross-view image pairs, on which output coordinates should a consistency loss be imposed?<n>We show that consistency applied to the covariant block is provably harmful, shrinking legitimate view-specific content to a fraction $1/(1+4)$ of its true value, and we verify this shrinkage law in controlled experiments.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World action models (WAMs) jointly denoise future video frames and robot actions, and the video prior is expected to generalize their control. Camera viewpoint change remains one of their hardest perturbation axes. We study a question specific to this model class: when training with same-state cross-view image pairs, on which output coordinates should a consistency loss be imposed? The WAM denoising target mixes view-covariant coordinates, namely the predicted future scene, with view-invariant coordinates, namely the action chunk, future proprioception, and value. We show that consistency applied to the covariant block is provably harmful, shrinking legitimate view-specific content to a fraction $1/(1+4λ)$ of its true value, and we verify this shrinkage law in controlled experiments. Selective cross-view consistency (SCVC) therefore constrains only the invariant block, requires no camera labels, extrinsics, depth, or view synthesis at training or test time, and leaves the deployment interface unchanged. We introduce a carve-and-hold-out evaluation protocol on the LIBERO-Plus camera track that separates a distribution-matched ceiling from genuine interpolation and extrapolation to held-out viewpoints, with a matched pair-trained control isolating the effect of the consistency term from pair exposure. On held-out orbital viewpoints beyond the training envelope, SCVC improves closed-loop success over the matched control by 12.2 points (95% CI [7.4, 17.0]; +15.5, CI [11.7, 19.4], under an independent second seed) -- an effect two further camera axes replicate -- while interpolation within the envelope shows no gain in either seed (-1.2 and -4.3 points) and in-distribution competence is preserved (-0.6, -0.2). We also report a cross-backbone audit showing that published camera-robustness numbers are confounded by wrist-camera pose stability.

Related papers

• AcrossWAM1.0:A Modular Latent World-Action Stack for Compact Robot Policies [14.556446786016787]
  We introduce AcrossWAM1.0, a modularization and scaling study of LaWAM's latent world-action stack.<n>AcrossWAM1.0 contributes an auditable software and evaluation boundary for compact latent world-action policies.
  arXiv  Detail & Related papers  (2026-08-30T18:09:17Z)
• Zero-Shot Traffic Accident Detection via a Coarse-to-Fine VLM-Tracking Pipeline [0.10874100424278173]
  ACCIDENT @ CVPR benchmark evaluates exactly this joint prediction under a strict constraint.<n>We introduce a training-free, two-pass coarse-to-fine pipeline that pairs a frozen Qwen3-VL-32B-Instruct vision-language model with YOLO11x object detection and BoT-SORT tracking.<n>On the official 2,027-clip real-CCTV test set, our system achieves a three-way harmonic mean score of 0.504.
  arXiv  Detail & Related papers  (2026-08-09T19:08:56Z)
• SC3-Eval: Evaluating Robot Foundation Models via Self-Consistent Video Generation [73.2142090645987]
  SC3-Eval is a self-consistent video generation recipe that adapts a pre-trained video foundation model into an accurate policy evaluator.<n>We demonstrate SC3-Eval rollouts reproduce the failure modes that policies exhibit in real-world rollouts, supporting fine-grained diagnostic comparison.
  arXiv  Detail & Related papers  (2026-06-17T02:15:46Z)
• ConfCtrl: Enabling Precise Camera Control in Video Diffusion via Confidence-Aware Interpolation [24.89894187462497]
  ConfCtrl is a confidence-aware video framework that enables camera-guided diffusion models to follow prescribed poses while completing unseen regions.<n>Experiments show that ConfCtrl produces geometrically consistent and visually plausible novel views, effectively reconstructing occluded regions under large viewpoint changes.
  arXiv  Detail & Related papers  (2026-03-10T15:44:48Z)
• PoseDiff: A Unified Diffusion Model Bridging Robot Pose Estimation and Video-to-Action Control [67.17998939712326]
  We present PoseDiff, a conditional diffusion model that unifies robot state estimation and control within a single framework.<n>At its core, PoseDiff maps raw visual observations into structured robot states-such as 3D keypoints or joint angles-from a single RGB image.<n>Building upon this foundation, PoseDiff extends naturally to video-to-action inverse dynamics.
  arXiv  Detail & Related papers  (2025-09-29T10:55:48Z)
• Sparse BEV Fusion with Self-View Consistency for Multi-View Detection and Tracking [15.680801582969393]
  We propose SCFusion, a framework that combines three techniques to improve multi-view feature integration.<n> SCFusion achieves state-of-the-art performance, reaching an IDF1 score of 95.9% on WildTrack and a MODP of 89.2% on MultiviewX.
  arXiv  Detail & Related papers  (2025-09-10T09:06:41Z)
• CamI2V: Camera-Controlled Image-to-Video Diffusion Model [11.762824216082508]
  Integrated camera pose is a user-friendly and physics-informed condition in video diffusion models, enabling precise camera control.<n>We identify one of the key challenges as effectively modeling noisy cross-frame interactions to enhance geometry consistency and camera controllability.<n>We innovatively associate the quality of a condition with its ability to reduce uncertainty and interpret noisy cross-frame features as a form of noisy condition.
  arXiv  Detail & Related papers  (2024-10-21T12:36:27Z)
• HVC-Net: Unifying Homography, Visibility, and Confidence Learning for Planar Object Tracking [5.236567998857959]
  We present a unified convolutional neural network (CNN) model that jointly considers homography, visibility, and confidence. Our approach outperforms the state-of-the-art methods on public POT and TMT datasets.
  arXiv  Detail & Related papers  (2022-09-19T11:11:56Z)
• Multi-View Correlation Consistency for Semi-Supervised Semantic Segmentation [59.34619548026885]
  Semi-supervised semantic segmentation needs rich and robust supervision on unlabeled data. We propose a view-coherent data augmentation strategy that guarantees pixel-pixel correspondence between different views. In a series of semi-supervised settings on two datasets, we report competitive accuracy compared with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-08-17T17:59:11Z)
• Hybrid Tracker with Pixel and Instance for Video Panoptic Segmentation [50.62685357414904]
  Video Panoptic coefficient (VPS) aims to generate coherent panoptic segmentation and track the identities of all pixels across video frames. We present HybridTracker, a lightweight and joint tracking model attempting to eliminate the limitations of the single tracker. Comprehensive experiments show that HybridTracker achieves superior performance than state-of-the-art methods on Cityscapes-VPS and VIPER datasets.
  arXiv  Detail & Related papers  (2022-03-02T16:21:55Z)
• Cross-Camera Feature Prediction for Intra-Camera Supervised Person Re-identification across Distant Scenes [70.30052164401178]
  Person re-identification (Re-ID) aims to match person images across non-overlapping camera views. ICS-DS Re-ID uses cross-camera unpaired data with intra-camera identity labels for training. Cross-camera feature prediction method to mine cross-camera self supervision information. Joint learning of global-level and local-level features forms a global-local cross-camera feature prediction scheme.
  arXiv  Detail & Related papers  (2021-07-29T11:27:50Z)
• Intra-Inter Camera Similarity for Unsupervised Person Re-Identification [50.85048976506701]
  We study a novel intra-inter camera similarity for pseudo-label generation. We train our re-id model in two stages with intra-camera and inter-camera pseudo-labels, respectively. This simple intra-inter camera similarity produces surprisingly good performance on multiple datasets.
  arXiv  Detail & Related papers  (2021-03-22T08:29:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.