Geometrically Plausible Object Pose Refinement using Differentiable Simulation

• URL: http://arxiv.org/abs/2603.20992v1
• Date: Sun, 22 Mar 2026 00:24:14 GMT
• Title: Geometrically Plausible Object Pose Refinement using Differentiable Simulation
• Abstract summary: State-of-the-art object pose estimation methods are prone to generating geometrically infeasible pose hypotheses.<n>We propose a multi-modal pose refinement approach that combines differentiable physics simulation, differentiable rendering and visuo-tactile sensing.
• Score: 2.833580209528223
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State-of-the-art object pose estimation methods are prone to generating geometrically infeasible pose hypotheses. This problem is prevalent in dexterous manipulation, where estimated poses often intersect with the robotic hand or are not lying on a support surface. We propose a multi-modal pose refinement approach that combines differentiable physics simulation, differentiable rendering and visuo-tactile sensing to optimize object poses for both spatial accuracy and physical consistency. Simulated experiments show that our approach reduces the intersection volume error between the object and robotic hand by 73\% when the initial estimate is accurate and by over 87\% under high initial uncertainty, significantly outperforming standard ICP-based baselines. Furthermore, the improvement in geometric plausibility is accompanied by a concurrent reduction in translation and orientation errors. Achieving pose estimation that is grounded in physical reality while remaining faithful to multi-modal sensor inputs is a critical step toward robust in-hand manipulation.

Related papers

• Differentiable Mesh State Estimation via Factor Graph Inference for Deformable Object Reconstruction [35.217259059423775]
  Estimating deformable object states remains a fundamental challenge in robotics and simulation.<n>We propose a novel factor graph-based framework for probabilistic mesh state estimation of deformable objects.
  arXiv  Detail & Related papers  (2026-09-15T06:07:15Z)
• PAD-Hand: Physics-Aware Diffusion for Hand Motion Recovery [8.428762763310232]
  We propose a physics-aware conditional diffusion framework that refines noisy pose sequences into physically plausible hand motion.<n>Building on a MeshCNN-Transformer backbone, we formulate Euler-Lagrange dynamics for articulated hands.<n>Our method produces per-joint, per-time variances that measure physics consistency and offer interpretable variance maps.
  arXiv  Detail & Related papers  (2026-03-27T04:39:21Z)
• Research on a Camera Position Measurement Method based on a Parallel Perspective Error Transfer Model [9.335726811337617]
  Camera pose estimation from sparse correspondences is a fundamental problem in computer vision.<n>We present a geometric error propagation framework for camera pose estimation based on a parallel perspective approximation.
  arXiv  Detail & Related papers  (2026-02-08T09:40:14Z)
• Robust Rigid Body Assembly via Contact-Implicit Optimal Control with Exact Second-Order Derivatives [0.9079873854272275]
  This paper proposes a sample-efficient robust optimal control approach for the determination of assembly motions.<n>A differentiable physics simulation is constructed that provides second-order analytic derivatives to the numerical solver.<n>The solution of the physics simulation problem is made differentiable by using smoothing inspired by interior-point methods applied to both the collision detection and the contact resolution problem.
  arXiv  Detail & Related papers  (2026-01-30T11:21:20Z)
• BayesSDF: Surface-Based Laplacian Uncertainty Estimation for 3D Geometry with Neural Signed Distance Fields [0.0]
  BayesSDF is a novel probabilistic framework for uncertainty estimation in neural implicit 3D representations.<n>By enabling surface-aware uncertainty quantification, BayesSDF lays the groundwork for more robust, interpretable, and actionable 3D perception systems.
  arXiv  Detail & Related papers  (2025-07-08T03:21:12Z)
• PhyRecon: Physically Plausible Neural Scene Reconstruction [81.73129450090684]
  We introduce PHYRECON, the first approach to leverage both differentiable rendering and differentiable physics simulation to learn implicit surface representations. Central to this design is an efficient transformation between SDF-based implicit representations and explicit surface points. Our results also exhibit superior physical stability in physical simulators, with at least a 40% improvement across all datasets.
  arXiv  Detail & Related papers  (2024-04-25T15:06:58Z)
• DeepSimHO: Stable Pose Estimation for Hand-Object Interaction via Physics Simulation [81.11585774044848]
  We present DeepSimHO, a novel deep-learning pipeline that combines forward physics simulation and backward gradient approximation with a neural network. Our method noticeably improves the stability of the estimation and achieves superior efficiency over test-time optimization.
  arXiv  Detail & Related papers  (2023-10-11T05:34:36Z)
• Poseur: Direct Human Pose Regression with Transformers [119.79232258661995]
  We propose a direct, regression-based approach to 2D human pose estimation from single images. Our framework is end-to-end differentiable, and naturally learns to exploit the dependencies between keypoints. Ours is the first regression-based approach to perform favorably compared to the best heatmap-based pose estimation methods.
  arXiv  Detail & Related papers  (2022-01-19T04:31:57Z)
• Adaptive Surface Normal Constraint for Depth Estimation [102.7466374038784]
  We introduce a simple yet effective method, named Adaptive Surface Normal (ASN) constraint, to correlate the depth estimation with geometric consistency. Our method can faithfully reconstruct the 3D geometry and is robust to local shape variations, such as boundaries, sharp corners and noises.
  arXiv  Detail & Related papers  (2021-03-29T10:36:25Z)
• Category Level Object Pose Estimation via Neural Analysis-by-Synthesis [64.14028598360741]
  In this paper we combine a gradient-based fitting procedure with a parametric neural image synthesis module. The image synthesis network is designed to efficiently span the pose configuration space. We experimentally show that the method can recover orientation of objects with high accuracy from 2D images alone.
  arXiv  Detail & Related papers  (2020-08-18T20:30:47Z)
• Scalable Differentiable Physics for Learning and Control [99.4302215142673]
  Differentiable physics is a powerful approach to learning and control problems that involve physical objects and environments. We develop a scalable framework for differentiable physics that can support a large number of objects and their interactions.
  arXiv  Detail & Related papers  (2020-07-04T19:07:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.