Constraint-Preserving Data Generation for Visuomotor Policy Learning

• URL: http://arxiv.org/abs/2508.03944v1
• Date: Tue, 05 Aug 2025 22:20:02 GMT
• Title: Constraint-Preserving Data Generation for Visuomotor Policy Learning
• Abstract summary: We use a single expert trajectory to generate robot demonstrations containing novel object geometries and poses.<n>These generated demonstrations are used to train closed-loop visuomotor policies that transfer zero-shot to the real world.
• Score: 46.576295441652405
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large-scale demonstration data has powered key breakthroughs in robot manipulation, but collecting that data remains costly and time-consuming. We present Constraint-Preserving Data Generation (CP-Gen), a method that uses a single expert trajectory to generate robot demonstrations containing novel object geometries and poses. These generated demonstrations are used to train closed-loop visuomotor policies that transfer zero-shot to the real world and generalize across variations in object geometries and poses. Similar to prior work using pose variations for data generation, CP-Gen first decomposes expert demonstrations into free-space motions and robot skills. But unlike those works, we achieve geometry-aware data generation by formulating robot skills as keypoint-trajectory constraints: keypoints on the robot or grasped object must track a reference trajectory defined relative to a task-relevant object. To generate a new demonstration, CP-Gen samples pose and geometry transforms for each task-relevant object, then applies these transforms to the object and its associated keypoints or keypoint trajectories. We optimize robot joint configurations so that the keypoints on the robot or grasped object track the transformed keypoint trajectory, and then motion plan a collision-free path to the first optimized joint configuration. Experiments on 16 simulation tasks and four real-world tasks, featuring multi-stage, non-prehensile and tight-tolerance manipulation, show that policies trained using CP-Gen achieve an average success rate of 77%, outperforming the best baseline that achieves an average of 50%.

Related papers

• Improving Imitation Learning Efficiency for Manipulation through Geometric Prior Pretraining [21.64711515041526]
  In this study, we construct a geometric visual pretraining dataset in which each scene contains only a plane, an object, and a hand.<n>We evaluate this geometric prior using ACT on three simulated robots across five manipulation tasks each, as well as on three real-world robot tasks.
  arXiv  Detail & Related papers  (2026-09-11T11:19:06Z)
• Iterative Grasp Pose Refinement: A Deep Reinforcement Learning Approach for 2D Vision [2.5899040911480182]
  This work proposes a reinforcement learning-based framework for robotic grasp refinement.<n>A geometric-based algorithm generates initial grasp candidates, which are iteratively refined by the proposed framework.<n>Experiments conducted on 300 objects from the Dex-Net dataset demonstrate the framework's effectiveness.
  arXiv  Detail & Related papers  (2026-08-18T10:44:58Z)
• PRISM: Personalized Robotic Dataset Generation via Image-based Scene and Motion Synthesis [5.244037702157958]
  PRISM is an end-to-end pipeline that generates personalized robotic datasets from a single image and a natural-language instruction.<n> PRISM constructs digital cousin scenes that are semantically and geometrically aligned with the user environment yet diverse at the instance level, and synthesizes executable demonstrations without human teleoperation.
  arXiv  Detail & Related papers  (2026-07-06T10:00:47Z)
• QDTraj: Exploration of Diverse Trajectory Primitives for Articulated Objects Robotic Manipulation [2.7261369763665377]
  This paper presents a method that enables a robot to manipulate a wide spectrum of articulated objects.<n>We automatically generate different robot low-level trajectory primitives to manipulate given object articulations.<n>QDTraj generates at least 5 times more diverse trajectories for both hinge and slider activation tasks.
  arXiv  Detail & Related papers  (2026-04-24T13:45:04Z)
• RoboGene: Boosting VLA Pre-training via Diversity-Driven Agentic Framework for Real-World Task Generation [37.52152452548065]
  RoboGene is an agentic framework designed to automate the generation of diverse, physically plausible manipulation tasks.<n>We conduct extensive quantitative analysis and large-scale real-world experiments, collecting datasets of 18k trajectories.<n>Results demonstrate that RoboGene significantly outperforms state-of-the-art foundation models.
  arXiv  Detail & Related papers  (2026-02-18T13:29:43Z)
• Generalizable Geometric Prior and Recurrent Spiking Feature Learning for Humanoid Robot Manipulation [90.90219129619344]
  This paper presents a novel R-prior-S, Recurrent Geometric-priormodal Policy with Spiking features.<n>To ground high-level reasoning in physical reality, we leverage lightweight 2D geometric inductive biases.<n>For the data efficiency issue in robotic action generation, we introduce a Recursive Adaptive Spiking Network.
  arXiv  Detail & Related papers  (2026-01-13T23:36:30Z)
• Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Control [72.00655365269]
  We present RoboMaster, a novel framework that models inter-object dynamics through a collaborative trajectory formulation.<n>Unlike prior methods that decompose objects, our core is to decompose the interaction process into three sub-stages: pre-interaction, interaction, and post-interaction.<n>Our method outperforms existing approaches, establishing new state-of-the-art performance in trajectory-controlled video generation for robotic manipulation.
  arXiv  Detail & Related papers  (2025-06-02T17:57:06Z)
• P3-PO: Prescriptive Point Priors for Visuo-Spatial Generalization of Robot Policies [19.12762500264209]
  Prescriptive Point Priors for Policies or P3-PO is a novel framework that constructs a unique state representation of the environment.<n>P3-PO exhibits 58% and 80% gains across tasks for new object instances and more cluttered environments respectively.
  arXiv  Detail & Related papers  (2024-12-09T18:59:42Z)
• Keypoint Abstraction using Large Models for Object-Relative Imitation Learning [78.92043196054071]
  Generalization to novel object configurations and instances across diverse tasks and environments is a critical challenge in robotics. Keypoint-based representations have been proven effective as a succinct representation for essential object capturing features. We propose KALM, a framework that leverages large pre-trained vision-language models to automatically generate task-relevant and cross-instance consistent keypoints.
  arXiv  Detail & Related papers  (2024-10-30T17:37:31Z)
• Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation [65.46610405509338]
  We seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation. Our framework,Track2Act predicts tracks of how points in an image should move in future time-steps based on a goal. We show that this approach of combining scalably learned track prediction with a residual policy enables diverse generalizable robot manipulation.
  arXiv  Detail & Related papers  (2024-05-02T17:56:55Z)
• Toward a Plug-and-Play Vision-Based Grasping Module for Robotics [0.0]
  This paper introduces a vision-based grasping framework that can easily be transferred across multiple manipulators. The framework generates diverse repertoires of open-loop grasping trajectories, enhancing adaptability while maintaining a diversity of grasps.
  arXiv  Detail & Related papers  (2023-10-06T16:16:00Z)
• Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
  We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models.<n>Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning.<n>Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
  arXiv  Detail & Related papers  (2023-06-09T07:22:12Z)
• Deep Reinforcement Learning Based on Local GNN for Goal-conditioned Deformable Object Rearranging [1.807492010338763]
  Object rearranging is one of the most common deformable manipulation tasks, where the robot needs to rearrange a deformable object into a goal configuration. Previous studies focus on designing an expert system for each specific task by model-based or data-driven approaches. We design a local GNN (Graph Neural Network) based learning method, which utilizes two representation graphs to encode keypoints detected from images. Our framework is effective in multiple 1-D (rope, rope ring) and 2-D (cloth) rearranging tasks in simulation and can be easily transferred to a real robot by fine-tuning a keypoint detector
  arXiv  Detail & Related papers  (2023-02-21T05:21:26Z)
• Efficient and Robust Training of Dense Object Nets for Multi-Object Robot Manipulation [8.321536457963655]
  We propose a framework for robust and efficient training of Dense Object Nets (DON) We focus on training with multi-object data instead of singulated objects, combined with a well-chosen augmentation scheme. We demonstrate the robustness and accuracy of our proposed framework on a real-world robotic grasping task.
  arXiv  Detail & Related papers  (2022-06-24T08:24:42Z)
• MetaGraspNet: A Large-Scale Benchmark Dataset for Vision-driven Robotic Grasping via Physics-based Metaverse Synthesis [78.26022688167133]
  We present a large-scale benchmark dataset for vision-driven robotic grasping via physics-based metaverse synthesis. The proposed dataset contains 100,000 images and 25 different object types. We also propose a new layout-weighted performance metric alongside the dataset for evaluating object detection and segmentation performance.
  arXiv  Detail & Related papers  (2021-12-29T17:23:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.