Pose6DAug: Physically Plausible Multi-view Object Swapping for Robot Data Augmentation

• URL: http://arxiv.org/abs/2606.20118v2
• Date: Fri, 19 Jun 2026 06:39:01 GMT
• Title: Pose6DAug: Physically Plausible Multi-view Object Swapping for Robot Data Augmentation
• Abstract summary: We introduce Pose6DAug, a failure-driven data augmentation framework.<n>We turn a policy's successful episodes into targeted demonstrations for its failure modes.<n>Our method operates directly in 3D, anchoring the target object with an explicit mesh driven by a temporally coherent 6D pose trajectory.
• Score: 52.61098793949261
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language-action (VLA) policies have shown strong potential for general-purpose manipulation, yet they often fail on novel, out-of-distribution objects whose appearance or geometry deviates from the training distribution. The standard remedy is to collect multi-view teleoperation data for every failure case, but this scales poorly in both cost and time. We introduce Pose6DAug, a failure-driven data augmentation framework that turns a policy's own successful episodes into targeted demonstrations for its failure modes, without any new data collection. Our key insight is that each successful episode already encodes a physically valid action trajectory together with calibrated multi-view observations. By swapping only the manipulated object while preserving this trajectory, we obtain new and physically grounded demonstrations. However, naive 2D video editing breaks multi-view consistency and physical plausibility, particularly under heavy occlusion and egocentric viewpoints. Our method instead operates directly in 3D, anchoring the target object with an explicit mesh driven by a temporally coherent 6D pose trajectory, ensuring geometrically consistent renderings across all camera views. Fine-tuning a VLA on data augmented by our method improves success rates by 16.5% relative to the state-of-the-art baseline on novel objects, while preserving in-distribution performance. These results show that multi-view and physically consistent augmentation is a practical path to scalable VLA generalization.

Related papers

• LAWM-3D: Learning 3D-Aware Latent Actions from Human Videos for Generalizable Robot World Models [64.11117186468533]
  World models enable agents to perform forward rollout and planning without real-world interaction.<n>Most existing LAMs rely on single-view inputs and operate primarily in 2D pixel space.<n>We propose LAWM-3D, which introduces three tightly coupled key designs for learning 3D-aware latent actions.
  arXiv  Detail & Related papers  (2026-08-06T07:47:01Z)
• From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data [71.22409934108924]
  Human videos are abundant and capture rich interactions, providing diverse semantic and physical cues for real-world manipulation.<n>This survey provides a unified view of how human videos are transformed into effective knowledge for Vision-Language-Action (VLA) models.<n>We highlight three key open challenges in this area: structuring unstructured videos into training-ready episodes, grounding video-derived supervision into robot-executable actions under embodiment and viewpoint heterogeneity, and designing evaluation protocols that better predict real-world deployment performance and transfer efficiency.
  arXiv  Detail & Related papers  (2026-05-18T06:19:16Z)
• Learning Action Manifold with Multi-view Latent Priors for Robotic Manipulation [67.16145181790522]
  This paper tackles spatial perception and manipulation challenges in Vision-Language-Action (VLA) models.<n>We propose a Geometry-Guided Gated Transformer (G3T) that aligns multi-view features under 3D geometric guidance.<n>To improve action learning efficiency, we introduce Action Manifold Learning (AML), which directly predicts actions on the valid action manifold.
  arXiv  Detail & Related papers  (2026-05-12T09:21:29Z)
• Universal Pose Pretraining for Generalizable Vision-Language-Action Policies [83.39008378156647]
  Existing Vision-Language-Action (VLA) models often suffer from feature collapse and low training efficiency.<n>We propose Pose-VLA, a decoupled paradigm that separates VLA training into a pre-training phase for extracting universal 3D spatial priors.<n>Our framework follows a two-stage pre-training pipeline, establishing fundamental spatial grounding via poses followed by motion alignment.
  arXiv  Detail & Related papers  (2026-02-23T11:00:08Z)
• PoseDiff: A Unified Diffusion Model Bridging Robot Pose Estimation and Video-to-Action Control [67.17998939712326]
  We present PoseDiff, a conditional diffusion model that unifies robot state estimation and control within a single framework.<n>At its core, PoseDiff maps raw visual observations into structured robot states-such as 3D keypoints or joint angles-from a single RGB image.<n>Building upon this foundation, PoseDiff extends naturally to video-to-action inverse dynamics.
  arXiv  Detail & Related papers  (2025-09-29T10:55:48Z)
• One-shot Video Imitation via Parameterized Symbolic Abstraction Graphs [8.872100864022675]
  We propose to interpret video demonstrations through Symbolicized Abstraction Graphs (PSAG) We further ground denote geometric constraints through simulation to estimate non-geometric, visually imperceptible attributes. Our approach has been validated across a range of tasks, such as Cutting Avocado, Cutting Vegetable, Pouring Liquid, Rolling Dough, and Slicing Pizza.
  arXiv  Detail & Related papers  (2024-08-22T18:26:47Z)
• Towards Robust and Smooth 3D Multi-Person Pose Estimation from Monocular Videos in the Wild [10.849750765175754]
  POTR-3D is a sequence-to-sequence 2D-to-3D lifting model for 3DMPPE. It robustly generalizes to diverse unseen views, robustly recovers the poses against heavy occlusions, and reliably generates more natural and smoother outputs.
  arXiv  Detail & Related papers  (2023-09-15T06:17:22Z)
• Learn the Force We Can: Enabling Sparse Motion Control in Multi-Object Video Generation [26.292052071093945]
  We propose an unsupervised method to generate videos from a single frame and a sparse motion input. Our trained model can generate unseen realistic object-to-object interactions. We show that YODA is on par with or better than state of the art video generation prior work in terms of both controllability and video quality.
  arXiv  Detail & Related papers  (2023-06-06T19:50:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.