Real2Edit2Real: Generating Robotic Demonstrations via a 3D Control Interface

• URL: http://arxiv.org/abs/2512.19402v1
• Date: Mon, 22 Dec 2025 13:53:25 GMT
• Title: Real2Edit2Real: Generating Robotic Demonstrations via a 3D Control Interface
• Abstract summary: We present Real2Edit2Real, a framework that generates new demonstrations by bridging 3D editability with 2D visual data through a 3D control interface.<n>Our approach first reconstructs scene geometry from multi-view RGB observations with a metric-scale 3D reconstruction model.<n>We propose a multi-conditional video generation model guided by depth as the primary control signal, together with action, edge, and ray maps, to synthesize spatially augmented multi-view manipulation videos.
• Score: 17.378883024907648
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent progress in robot learning has been driven by large-scale datasets and powerful visuomotor policy architectures, yet policy robustness remains limited by the substantial cost of collecting diverse demonstrations, particularly for spatial generalization in manipulation tasks. To reduce repetitive data collection, we present Real2Edit2Real, a framework that generates new demonstrations by bridging 3D editability with 2D visual data through a 3D control interface. Our approach first reconstructs scene geometry from multi-view RGB observations with a metric-scale 3D reconstruction model. Based on the reconstructed geometry, we perform depth-reliable 3D editing on point clouds to generate new manipulation trajectories while geometrically correcting the robot poses to recover physically consistent depth, which serves as a reliable condition for synthesizing new demonstrations. Finally, we propose a multi-conditional video generation model guided by depth as the primary control signal, together with action, edge, and ray maps, to synthesize spatially augmented multi-view manipulation videos. Experiments on four real-world manipulation tasks demonstrate that policies trained on data generated from only 1-5 source demonstrations can match or outperform those trained on 50 real-world demonstrations, improving data efficiency by up to 10-50x. Moreover, experimental results on height and texture editing demonstrate the framework's flexibility and extensibility, indicating its potential to serve as a unified data generation framework.

Related papers

• R2RDreamer: 3D-aware Data Augmentation for Spatially-generalized 2D Manipulation Policies [86.2249156068836]
  R2RDreamer is a real-to-real demonstration augmentation framework.<n>It preserves the geometric consistency of 3D action-observation editing while moving visual completion to 2D video space.
  arXiv  Detail & Related papers  (2026-06-15T17:56:19Z)
• Generative Texture Diversification of 3D Pedestrians for Robust Autonomous Driving Perception [0.0]
  In this paper, we propose a method for scaling variability in 3D pedestrian assets for synthetic scene generation.<n>We generate multiple distinct pedestrian instances by synthesizing diverse facial textures and identity-level appearance variations.<n>Using the assets, we construct synthetic datasets and study the impact of mixing real and synthetic data for RGB-based object detection.
  arXiv  Detail & Related papers  (2026-05-13T16:35:50Z)
• DynaRend: Learning 3D Dynamics via Masked Future Rendering for Robotic Manipulation [52.136378691610524]
  We present DynaRend, a representation learning framework that learns 3D-aware and dynamics-informed triplane features.<n>By pretraining on multi-view RGB-D video data, DynaRend jointly captures spatial geometry, future dynamics, and task semantics in a unified triplane representation.<n>We evaluate DynaRend on two challenging benchmarks, RLBench and Colosseum, demonstrating substantial improvements in policy success rate, generalization to environmental perturbations, and real-world applicability across diverse manipulation tasks.
  arXiv  Detail & Related papers  (2025-10-28T10:17:11Z)
• R2RGEN: Real-to-Real 3D Data Generation for Spatially Generalized Manipulation [74.41728218960465]
  We propose a real-to-real 3D data generation framework (R2RGen) that directly augments the pointcloud observation-action pairs to generate real-world data.<n>R2RGen substantially enhances data efficiency on extensive experiments and demonstrates strong potential for scaling and application on mobile manipulation.
  arXiv  Detail & Related papers  (2025-10-09T17:55:44Z)
• EMMA: Generalizing Real-World Robot Manipulation via Generative Visual Transfer [35.27100635173712]
  Vision-language-action (VLA) models increasingly rely on diverse training data to achieve robust generalization.<n>We propose Embodied Manipulation Media Adaptation (EMMA), a VLA policy enhancement framework that integrates a generative data engine with an effective training pipeline.<n>DreamTransfer enables text-controlled visual editing of robot videos, transforming foreground, background, and lighting conditions without compromising 3D structure or geometrical plausibility.<n>AdaMix is a hard-sample-aware training strategy that dynamically reweights training batches to focus optimization on perceptually or kinematically challenging samples.
  arXiv  Detail & Related papers  (2025-09-26T14:34:44Z)
• Geometry-aware 4D Video Generation for Robot Manipulation [28.709339959536106]
  We propose a 4D video generation model that enforces multi-view 3D consistency of videos by supervising the model with cross-view pointmap alignment during training.<n>This geometric supervision enables the model to learn a shared 3D representation of the scene, allowing it to predict future video sequences from novel viewpoints.<n>Compared to existing baselines, our method produces more visually stable and spatially aligned predictions across multiple simulated and real-world robotic datasets.
  arXiv  Detail & Related papers  (2025-07-01T18:01:41Z)
• RoboPearls: Editable Video Simulation for Robot Manipulation [81.18434338506621]
  RoboPearls is an editable video simulation framework for robotic manipulation.<n>Built on 3D Gaussian Splatting (3DGS), RoboPearls enables the construction of photo-realistic, view-consistent simulations.<n>We conduct extensive experiments on multiple datasets and scenes, including RLBench, COLOSSEUM, Ego4D, Open X-Embodiment, and a real-world robot.
  arXiv  Detail & Related papers  (2025-06-28T05:03:31Z)
• DGS-LRM: Real-Time Deformable 3D Gaussian Reconstruction From Monocular Videos [52.46386528202226]
  We introduce the Deformable Gaussian Splats Large Reconstruction Model (DGS-LRM)<n>It is the first feed-forward method predicting deformable 3D Gaussian splats from a monocular posed video of any dynamic scene.<n>It achieves performance on par with state-of-the-art monocular video 3D tracking methods.
  arXiv  Detail & Related papers  (2025-06-11T17:59:58Z)
• WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild [53.288327629960364]
  We present a data-driven pipeline for efficient multi-hand reconstruction in the wild.<n>The proposed pipeline is composed of two components: a real-time fully convolutional hand localization and a high-fidelity transformer-based 3D hand reconstruction model.<n>Our approach outperforms previous methods in both efficiency and accuracy on popular 2D and 3D benchmarks.
  arXiv  Detail & Related papers  (2024-09-18T18:46:51Z)
• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.