URDF Synthesis from RGB-D Sequences via Differentiable Joint Inference and Energy-Consistent Verification

• URL: http://arxiv.org/abs/2606.18861v1
• Date: Wed, 17 Jun 2026 09:42:25 GMT
• Title: URDF Synthesis from RGB-D Sequences via Differentiable Joint Inference and Energy-Consistent Verification
• Abstract summary: KinemaForge is a constraint-driven pipeline that infers part-level shape, joint topology, and joint parameters from short RGB-D sequences.<n>It reduces the average joint-axis error from 4.52 degrees to 2.83 degrees (-37.4%) over the strongest geometric baseline.<n>KinemaForge yields URDFs whose closed-loop manipulation success rate improves by 14.6 percentage points over Ditto.
• Score: 2.538209532048867
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstructing simulation-ready digital twins of articulated objects from sensor observations remains constrained by two persistent gaps: (i) part-level geometric reconstruction is decoupled from kinematic-parameter estimation, and (ii) the recovered models often violate basic dynamic invariants such as energy conservation, leading to drift when the URDF is replayed in physics simulators. We present KinemaForge, a constraint-driven pipeline that jointly infers part-level shape, joint topology, and joint parameters from short RGB-D sequences and validates the result against an energy-consistent verifier built on differentiable rigid-body dynamics. The pipeline introduces three components: a kinematic constraint graph that encodes joint-part incidences as soft edges; a differentiable screw-axis solver that backpropagates from rendered observations through Featherstone's articulated-body algorithm to joint parameters; and an energy residual loss that penalises non-physical free responses of the reconstructed model. Across five PartNet-Mobility categories and an internal RGB-D benchmark, KinemaForge reduces the average joint-axis error from 4.52 degrees to 2.83 degrees (-37.4%) over the strongest geometric baseline (PARIS) and from 5.30 degrees to 2.83 degrees (-46.6%) over the interaction-based Ditto baseline, lowers long-horizon simulation drift by 64% (vs. PARIS) over 50 s rollouts, and yields URDFs whose closed-loop manipulation success rate improves by 14.6 percentage points over Ditto in our preliminary evaluation. Code and reconstruction data will be released upon acceptance.

Related papers

• ASTRA: Asynchronous Spatio-Temporal Reconstruction via Trajectory Alignment [86.35109211408489]
  We propose ASTRA, a framework that introduces 2D motion trajectories as explicit, texture-agnostic supervision for asynchronous dynamic reconstruction.<n>ASTRA preserves high-frequency spatial details and sustains strong robustness even under severe asynchrony with up to 25-frame offsets, achieving approximately 1.4 dB PSNR improvement, reducing temporal-offset MAE by 54.0%, and nearly quadrupling the synchronization success rate.
  arXiv  Detail & Related papers  (2026-08-03T10:06:45Z)
• GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis [94.20462261657525]
  GrainGS is a dynamic framework that combines a hierarchical anchor scaffold with per-Gaussian deformation.<n>It achieves high reconstruction quality, real-time novel view synthesis, and compact storage.
  arXiv  Detail & Related papers  (2026-07-23T15:52:35Z)
• SPARC-Net: A Spectral, Causality-Aware, and Hard-Constrained Physics-Informed Architecture for Stiff and Shock-Dominated Partial Differential Equations [0.0654051537828115]
  Physics-Informed Neural Networks (PINNs) provide a meshless approach for solving partial differential equations (PDEs)<n>PINNs suffer severe degradation in stiff and shock-dominated problems, where small PDE residuals can correspond to globally inaccurate solutions.<n>We show these failures are multi-causal, arising from the concurrent interplay of (i) spectral bias against sharp features, (ii) imbalanced multi-term optimization and loss-weight collapse, (iii) violation of temporal causality, and (iv) under-resolved collocation.<n>We present-Net, a unified architecture and training
  arXiv  Detail & Related papers  (2026-07-13T09:23:17Z)
• KPGrasp: Scalable Keypoint Flow Matching for Dexterous Grasp Generation [23.028638559051597]
  We present KPGrasp, a flow-matching framework that learns dexterous grasp priors from large-scale data.<n>Experiments on 20 diverse objects demonstrate that the pipeline can be deployed in a real-world setup.
  arXiv  Detail & Related papers  (2026-06-08T10:19:29Z)
• How Much Capacity Does EEG Denoising Need? Ultra-Compact Networks reveal Benchmark Saturation and Metric-Utility Gap [0.0]
  Deep learning EEG denoising architectures have scaled from tens of thousands to tens of millions of parameters, yet no prior study has isolated model capacity as the experimental variable.<n>We address both gaps by fixing architecture, loss, data split, and training recipe while sweeping only channel width from 1.05K to 40.26K parameters in a minimal depthwiseseparable convolutional U-Net.
  arXiv  Detail & Related papers  (2026-06-07T12:17:25Z)
• GARDEN: Gravity-Aligned Reconstruction of Disentangled ENvironments from RGB images [61.2170105836525]
  We propose an RGB-only framework that reformulates reconstruction as physically-grounded scene factorization and outputs a structured hybrid scene representation.<n>The resulting representation combines explicit rigid bodies with a decoupled background, enabling direct physics simulation while preserving visual realism.
  arXiv  Detail & Related papers  (2026-06-02T17:13:01Z)
• Factor Graph-Based Shape Estimation for Continuum Robots via Magnus Expansion [19.359841144204115]
  This letter combines the strength of both paradigms by estimating the coefficients of a low-dimensional Geometric Variable Strain (GVS) parameterization within a factor graph framework.<n>A novel kinematic factor, derived from the Magnus expansion of the strain field, encodes the closed-form rod geometry as a prior constraint linking the GVS strain coefficients to the backbone pose variables.<n>The resulting formulation yields a compact state vector amenable directly to model-based control, while retaining the continuum, probabilistic treatment and computational efficiency of factor graph inference.
  arXiv  Detail & Related papers  (2026-04-17T01:52:07Z)
• PoInit-of-View: Poisoning Initialization of Views Transfers Across Multiple 3D Reconstruction Systems [55.84012796735184]
  Poisoning input views of 3D reconstruction systems has been recently studied.<n>We propose PoInit-of-View, which optimize adversarial perturbations to intentionally introduce cross-view inconsistencies.<n> Experimental results demonstrate the effectiveness of our PoInit-of-View on diverse 3D reconstruction systems and datasets.
  arXiv  Detail & Related papers  (2026-04-17T00:48:12Z)
• End-to-End Spatial-Temporal Transformer for Real-time 4D HOI Reconstruction [74.31251139839047]
  THO is an end-to-end Spatial-Temporal Transformer that predicts human motion and coordinated object motion in a forward fashion from the given video and 3D template.<n> experiments demonstrate that THO operates at an inference speed of 31.5 FPS on a single 4090 GPU, achieving a >600x speedup over prior optimization-based methods.
  arXiv  Detail & Related papers  (2026-03-15T15:21:36Z)
• RLGF: Reinforcement Learning with Geometric Feedback for Autonomous Driving Video Generation [75.61028930882144]
  We identify and quantify this critical issue, demonstrating a significant performance gap in 3D object detection when using synthetic versus real data.<n>We introduce Reinforcement Learning with Geometric Feedback (RLGF), RLGF uniquely refines video diffusion models by incorporating rewards from specialized latent-space AD perception models.<n> RLGF substantially reduces geometric errors (e.g., VP error by 21%, Depth error by 57%) and dramatically improves 3D object detection mAP by 12.7%, narrowing the gap to real-data performance.
  arXiv  Detail & Related papers  (2025-09-20T02:23:36Z)
• Lightweight Deep Learning Framework for Accurate Particle Flow Energy Reconstruction [8.598010350935596]
  This paper systematically evaluates a deep learning reconstruction framework.<n>We design a hybrid loss function combining weighted mean squared with error structural similarity index.<n>We enhance the model's capability to capture cross-modaltemporal correlations and energy-displacement nonlinearities.
  arXiv  Detail & Related papers  (2024-10-08T11:49:18Z)
• PhyRecon: Physically Plausible Neural Scene Reconstruction [81.73129450090684]
  We introduce PHYRECON, the first approach to leverage both differentiable rendering and differentiable physics simulation to learn implicit surface representations. Central to this design is an efficient transformation between SDF-based implicit representations and explicit surface points. Our results also exhibit superior physical stability in physical simulators, with at least a 40% improvement across all datasets.
  arXiv  Detail & Related papers  (2024-04-25T15:06:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.