Beyond Viewpoint Generalization: What Multi-View Demonstrations Offer and How to Synthesize Them for Robot Manipulation?

• URL: http://arxiv.org/abs/2603.26757v1
• Date: Mon, 23 Mar 2026 15:01:15 GMT
• Title: Beyond Viewpoint Generalization: What Multi-View Demonstrations Offer and How to Synthesize Them for Robot Manipulation?
• Abstract summary: Multi-view demonstrations consistently improve single-view policy success and generalization.<n>Mechanistic analysis shows that multi-view learning promotes manipulation-relevant visual representations.<n>We propose RoboNVS, a geometry-aware self-supervised framework that synthesizes novel-view videos from monocular inputs.
• Score: 28.177241923955435
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Does multi-view demonstration truly improve robot manipulation, or merely enhance cross-view robustness? We present a systematic study quantifying the performance gains, scaling behavior, and underlying mechanisms of multi-view data for robot manipulation. Controlled experiments show that, under both fixed and randomized backgrounds, multi-view demonstrations consistently improve single-view policy success and generalization. Performance varies non-monotonically with view coverage, revealing effective regimes rather than a simple "more is better" trend. Notably, multi-view data breaks the scaling limitation of single-view datasets and continues to raise performance ceilings after saturation. Mechanistic analysis shows that multi-view learning promotes manipulation-relevant visual representations, better aligns the action head with the learned feature distribution, and reduces overfitting. Motivated by the importance of multi-view data and its scarcity in large-scale robotic datasets, as well as the difficulty of collecting additional viewpoints in real world settings, we propose RoboNVS, a geometry-aware self-supervised framework that synthesizes novel-view videos from monocular inputs. The generated data consistently improves downstream policies in both simulation and real-world environments.

Related papers

• Multi-Camera View Scaling for Data-Efficient Robot Imitation Learning [57.88781687419521]
  Multiple synchronized camera perspectives are used to generate pseudo-demonstrations from each expert trajectory.<n>Experiments in simulation and real-world manipulation tasks demonstrate significant gains in data efficiency and generalization.<n>Our results suggest that scaling camera views provides a practical and scalable solution for imitation learning.
  arXiv  Detail & Related papers  (2026-04-01T07:00:44Z)
• Data Analogies Enable Efficient Cross-Embodiment Transfer [72.88252238231269]
  Generalist robot policies are trained on demonstrations collected across a wide variety of robots, scenes, and viewpoints.<n>We ask: what form of demonstration data is most useful for enabling transfer across robot set-ups?<n>Our simulated experiments show that while perceptual shifts such as viewpoint benefit most from broad diversity, morphology shifts benefit far less from unstructured diversity.
  arXiv  Detail & Related papers  (2026-03-06T16:42:46Z)
• RoboVIP: Multi-View Video Generation with Visual Identity Prompting Augments Robot Manipulation [38.65727896970541]
  We build a scalable pipeline to curate a visual identity pool from large datasets.<n>Using our augmented manipulation data to train downstream vision-language-action and visuomotor policy models yields consistent performance gains in both simulation and real-robot settings.
  arXiv  Detail & Related papers  (2026-01-08T18:59:22Z)
• Invariance Co-training for Robot Visual Generalization [72.88252238231269]
  Reasoning from diverse observations is a fundamental capability for generalist robot policies to operate in a wide range of environments.<n>We show that co-training on diverse data improves performance by 18 percent over existing generative augmentation methods.
  arXiv  Detail & Related papers  (2025-12-04T20:08:46Z)
• EMMA: Generalizing Real-World Robot Manipulation via Generative Visual Transfer [35.27100635173712]
  Vision-language-action (VLA) models increasingly rely on diverse training data to achieve robust generalization.<n>We propose Embodied Manipulation Media Adaptation (EMMA), a VLA policy enhancement framework that integrates a generative data engine with an effective training pipeline.<n>DreamTransfer enables text-controlled visual editing of robot videos, transforming foreground, background, and lighting conditions without compromising 3D structure or geometrical plausibility.<n>AdaMix is a hard-sample-aware training strategy that dynamically reweights training batches to focus optimization on perceptually or kinematically challenging samples.
  arXiv  Detail & Related papers  (2025-09-26T14:34:44Z)
• View-Invariant Policy Learning via Zero-Shot Novel View Synthesis [26.231630397802785]
  We investigate how knowledge from large-scale visual data of the world may be used to address one axis of variation for generalizable manipulation: observational viewpoint.<n>We study single-image novel view synthesis models, which learn 3D-aware scene-level priors by rendering images of the same scene from alternate camera viewpoints.<n>For practical application to diverse robotic data, these models must operate zero-shot, performing view synthesis on unseen tasks and environments.
  arXiv  Detail & Related papers  (2024-09-05T16:39:21Z)
• Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs [61.143381152739046]
  We introduce Cambrian-1, a family of multimodal LLMs (MLLMs) designed with a vision-centric approach.<n>Our study uses LLMs and visual instruction tuning as an interface to evaluate various visual representations.<n>We provide model weights, code, supporting tools, datasets, and detailed instruction-tuning and evaluation recipes.
  arXiv  Detail & Related papers  (2024-06-24T17:59:42Z)
• What Makes Pre-Trained Visual Representations Successful for Robust Manipulation? [57.92924256181857]
  We find that visual representations designed for manipulation and control tasks do not necessarily generalize under subtle changes in lighting and scene texture. We find that emergent segmentation ability is a strong predictor of out-of-distribution generalization among ViT models.
  arXiv  Detail & Related papers  (2023-11-03T18:09:08Z)
• Visual Imitation Made Easy [102.36509665008732]
  We present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector. We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task.
  arXiv  Detail & Related papers  (2020-08-11T17:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.