PhysV2A: Reachability-Gated and Semantic-Mask-Constrained Feasibility Completion for Video-to-Robot Manipulation

• URL: http://arxiv.org/abs/2607.09365v1
• Date: Fri, 10 Jul 2026 12:43:00 GMT
• Title: PhysV2A: Reachability-Gated and Semantic-Mask-Constrained Feasibility Completion for Video-to-Robot Manipulation
• Abstract summary: Video-based manipulation provides object-centric motion priors from human demonstrations, generated videos, or RGB-D observations.<n>PhysV2A is a framework for converting video-derived 6D object motion into robot-executable manipulation trajectories.
• Score: 17.70834422785338
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video-based manipulation provides object-centric motion priors from human demonstrations, generated videos, or RGB-D observations, but such priors are typically embodiment-agnostic and cannot be directly executed by a specific robot. This paper presents \textbf{PhysV2A}, a reachability-gated and semantic-mask-constrained feasibility-completion framework for converting video-derived 6D object motion into robot-executable manipulation trajectories. The key idea is to treat grasp feasibility as trajectory-conditioned rather than local: each RGB-D-generated 6-DoF grasp candidate is rigidly coupled with the recovered object motion to form a grasp-conditioned TCP trajectory hypothesis. PhysV2A then performs hierarchical reachability-gated selection, where infeasible grasp--trajectory pairs are rejected by robot-centric kinematic checks and surviving candidates are ranked by downstream execution suitability. For the selected reachable trajectory, a VLM-assisted and rule-validated S-Mask identifies task-critical and relaxable Cartesian components, enabling semantic-mask-constrained manipulability refinement through redundancy-first optimization and bounded Cartesian relaxation. Real-robot experiments on four tabletop manipulation tasks show that PhysV2A improves task success over representative video-prior and IK-only baselines, reduces kinematic-feasibility failures, and produces better-conditioned trajectories with bounded semantic deviations.

Related papers

• RGB-D Video Generation for Improving Human-to-Robot Object Handover Prediction [51.76737991200891]
  Human-to-robot (H2R) object handover is a fundamental capability for human-robot collaboration, yet progress is hindered by the scarcity of large-scale, human-centric datasets.<n>We introduce Hand2Bot, an RGB-D video dataset that provides rich contextual information such as body posture and facial expressions.<n>We propose PassGen, a generative pipeline that synthesizes realistic handover sequences while ensuring hand-object consistency.
  arXiv  Detail & Related papers  (2026-08-13T09:55:59Z)
• KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Grounding [58.541098216700796]
  evaluating the physical consistency of embodied world models is a critical open challenge.<n>Existing frameworks almost exclusively rely on Inverse Dynamics Models(IDMs) for action extraction.<n>We present KineBench, an IDM-free closed-loop benchmark for EWMs.
  arXiv  Detail & Related papers  (2026-07-22T08:04:17Z)
• GenVid2Robot: From Video Generation to Robot Manipulation via Rigid-Geometric Consistency [19.566368780107613]
  GenVid2Robot is a framework that converts generated video motion into executable real-robot manipulation trajectories.<n>Experiments demonstrate that GenVid2Robot improves the reliability of generated-video-guided manipulation by grounding visual motion priors with sparse metric geometry.
  arXiv  Detail & Related papers  (2026-07-10T08:32:47Z)
• Synthetic-to-Real Translation for Class-Agnostic Motion Prediction [64.55842058684583]
  We explore the potential of synthetic-to-real translation for motion prediction (SRMP)<n>We propose a novel approach integrating a motion knowledge translation framework with two key components.<n>We present a physically-based pipeline for generating Motion4D, the first synthetic 4D LiDAR dataset tailored for SRMP research.
  arXiv  Detail & Related papers  (2026-07-07T14:19:22Z)
• From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation [88.39072412680633]
  We propose MoLA, a control-oriented interface that transforms imagined future videos into executable representations.<n>We evaluate our approach on simulated benchmarks and real-world robot manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-12T14:15:16Z)
• DexSynRefine: Synthesizing and Refining Human-Object Interaction Motion for Physically Feasible Dexterous Robot Actions [0.5599792629509227]
  We present DexSyn, a framework for learning dexterous manipulation from human-object interaction (HOI) data.<n>We also present a contact-and-orientation adaptation module that enables sim-to-real transfer from proprioceptive history.
  arXiv  Detail & Related papers  (2026-05-07T09:31:43Z)
• ViPS: Video-informed Pose Spaces for Auto-Rigged Meshes [55.32681167870698]
  Video-informed Pose Spaces (ViPS) is a feed-forward framework that discovers the latent distribution of valid articulations for auto-rigged meshes.<n>ViPS transfers generative video priors into a universal distribution over a given rig parameterization.<n>Our evaluations show that ViPS matches the performance of state-of-the-art methods trained on synthetic artist-created 4D data in both plausibility and diversity.
  arXiv  Detail & Related papers  (2026-04-19T21:21:11Z)
• EmboAlign: Aligning Video Generation with Compositional Constraints for Zero-Shot Manipulation [17.037238350207513]
  Video generative models (VGMs) pretrained on large-scale internet data can produce temporally coherent rollout videos.<n>We present method, a data-free framework that aligns VGM outputs with insight generated by vision-language models at inference time.<n>We evaluate method on six real-robot manipulation tasks requiring precise, constraint-sensitive execution.
  arXiv  Detail & Related papers  (2026-03-05T23:31:56Z)
• DRAW2ACT: Turning Depth-Encoded Trajectories into Robotic Demonstration Videos [24.681248200255975]
  Video models provide powerful real-world simulators for embodied AI but remain limited in controllability for robotic manipulation.<n>We present DRAW2ACT, a trajectory-conditioned video generation framework that extracts multiple representations from the input trajectory.<n>We show that DRAW2ACT achieves superior visual fidelity and consistency while yielding higher manipulation success rates compared to existing baselines.
  arXiv  Detail & Related papers  (2025-12-16T09:11:36Z)
• ReMoSPLAT: Reactive Mobile Manipulation Control on a Gaussian Splat [3.983697124458495]
  Reactive control can gracefully coordinate the motion of the base and the arm of a mobile manipulator.<n>We present ReMoSPLAT, a reactive controller based on a quadratic program formulation for mobile manipulation.
  arXiv  Detail & Related papers  (2025-12-10T13:52:08Z)
• Spatial-Temporal Graph Diffusion Policy with Kinematic Modeling for Bimanual Robotic Manipulation [88.83749146867665]
  Existing approaches learn a policy to predict a distant next-best end-effector pose.<n>They then compute the corresponding joint rotation angles for motion using inverse kinematics.<n>We propose Kinematics enhanced Spatial-TemporAl gRaph diffuser.
  arXiv  Detail & Related papers  (2025-03-13T17:48:35Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)
• Learning to Segment Rigid Motions from Two Frames [72.14906744113125]
  We propose a modular network, motivated by a geometric analysis of what independent object motions can be recovered from an egomotion field. It takes two consecutive frames as input and predicts segmentation masks for the background and multiple rigidly moving objects, which are then parameterized by 3D rigid transformations. Our method achieves state-of-the-art performance for rigid motion segmentation on KITTI and Sintel.
  arXiv  Detail & Related papers  (2021-01-11T04:20:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.