DragMesh-2: Physically Plausible Dexterous Hand-Object Interaction with Articulated Objects

• URL: http://arxiv.org/abs/2606.15133v1
• Date: Sat, 13 Jun 2026 05:47:14 GMT
• Title: DragMesh-2: Physically Plausible Dexterous Hand-Object Interaction with Articulated Objects
• Abstract summary: Dexterous interaction with articulated objects is important for household, assistive, and humanoid manipulation.<n>We present DragMesh-2, a contact-driven framework for dexterous interaction with articulated objects.<n>We also propose PICA, a physically informed contact-aware training mechanism.
• Score: 12.35353010124029
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Dexterous interaction with articulated objects is important for household, assistive, and humanoid manipulation, where multi-finger hands can provide compliant contact patterns beyond parallel-jaw grasping. However, articulated-object manipulation differs from static-object manipulation: the target part cannot be directly actuated, and its motion must emerge through sustained physical hand--handle contact. This makes the transition from object-centric articulated generation to hand-driven dexterous hand--object interaction non-trivial, since geometric trajectory replay or open-loop execution does not model the contact dynamics required to move the articulated part. Moreover, policies trained only for task completion under fixed dynamics can overfit nominal contact loads, especially without tactile or force feedback, and may degrade when the contact load changes. To address these challenges, we present DragMesh-2, a contact-driven framework for dexterous interaction with articulated objects that extends articulated interaction from object-centric generation to hand-driven dexterous hand--object interaction, where articulated motion must arise through physical contact. We further propose PICA, a physically informed contact-aware training mechanism that injects physical signals into policy learning without tactile or force feedback, improving robustness and task success under changing contact loads. Finally, we conduct systematic evaluation across multiple damping conditions and articulated-object categories to study robustness under contact-load variation, and provide a pure-geometry dexterous interaction resource to support future loco-manipulation and humanoid hand--object interaction research. Across seven GAPartNet objects, DragMesh-2 achieves stronger robustness under contact-load variation than the compared methods while maintaining high task success across damping conditions.

Related papers

• AdaDexGrasp: Adaptive Dexterous Grasping via 3D Visuo-Tactile Representation Fusion [60.79507611737292]
  Humans achieve stable and adaptive grasps by seamlessly integrating visual perception and tactile feedback.<n>Existing robotic grasping approaches rely on visual inputs and lack mechanisms for tactile-guided adaptation after contact.<n>We propose a unified visuo-tactile-fusion grasping framework that integrates grasp generation, feasibility prediction, and adaptive refinement.
  arXiv  Detail & Related papers  (2026-08-06T16:29:11Z)
• GIRAF: Towards Generalizable Human Interactions with Articulated Objects [50.52795413484619]
  We present a text-conditioned diffusion model that addresses the challenge of Synthesizing realistic full-body human interactions with articulated objects.<n>Our method demonstrated strong generalization to unseen object configurations, surpassing current state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-07-08T19:30:44Z)
• Transferring Contact, Not Just Motion: Compliant Grasping Across Dexterous Hands [0.01268579273097071]
  Dexterous grasping depends on contact regulation, not motion alone.<n>Existing cross-embodiment dexterous policies unify motion through retargeted hand poses or latent actions, but force feedback remains tied to each hand's sensing and actuation.<n>This work introduces a cross-embodiment force-position interface for contact-aware manipulation across heterogeneous dexterous hands.
  arXiv  Detail & Related papers  (2026-06-14T00:01:36Z)
• SECOND-Grasp: Semantic Contact-guided Dexterous Grasping [60.1519218638742]
  Second-Grasp (SEmantic CONtact-guided Dexterous Grasping) is a unified framework that enables robotic hands to adjust grasping strategies based on semantic reasoning.<n>We show that our approach consistently outperforms baselines in lifting success rate on both seen and unseen categories.
  arXiv  Detail & Related papers  (2026-05-13T07:37:00Z)
• Tele-Catch: Adaptive Teleoperation for Dexterous Dynamic 3D Object Catching [30.78650584795827]
  We present Tele-Catch, a systematic framework for dexterous hand teleoperation in dynamic object catching.<n>At its core, we design DAIM, a dynamics-aware adaptive integration mechanism that realizes shared autonomy by fusing glove-based teleoperation signals into the diffusion policy denoising process.<n>Experiments demonstrate that Tele-Catch significantly improves accuracy and robustness in dynamic catching tasks, while also exhibiting consistent gains across distinct dexterous hand embodiments and previously unseen object categories.
  arXiv  Detail & Related papers  (2026-03-30T13:34:14Z)
• Unleashing Guidance Without Classifiers for Human-Object Interaction Animation [54.24065561632133]
  Prior diffusion-based approaches often rely on hand-crafted contact priors or human-imposed kinematic constraints to improve contact quality.<n>We propose a data-driven alternative in which guidance emerges from the denoising pace itself, reducing dependence on manually designed priors.<n>We find that this data-driven guidance is inherently contact-aware, and can be enhanced when training is augmented with a broad spectrum of synthetic object geometries.
  arXiv  Detail & Related papers  (2026-03-26T17:59:34Z)
• Towards Realistic Hand-Object Interaction with Gravity-Field Based Diffusion Bridge [23.484478357333856]
  We formulate hand-object interaction as an attraction-driven process and propose a Gravity-Field Based Diffusion Bridge (GravityDB) to simulate interactions between a deformable hand surface and rigid objects.<n>Our approach effectively resolves the aforementioned issues by generating physically plausible interactions that are free of interpenetration, ensure stable grasping, and capture realistic hand deformations.
  arXiv  Detail & Related papers  (2025-09-03T08:15:19Z)
• AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touch [9.606323817785114]
  We present AnyRotate, a system for gravity-invariant multi-axis in-hand object rotation using dense featured sim-to-real touch. Our formulation allows the training of a unified policy to rotate unseen objects about arbitrary rotation axes in any hand direction. Rich multi-fingered tactile sensing can detect unstable grasps and provide a reactive behavior that improves the robustness of the policy.
  arXiv  Detail & Related papers  (2024-05-12T22:51:35Z)
• Twisting Lids Off with Two Hands [82.21668778600414]
  We show how policies trained in simulation can be effectively and efficiently transferred to the real world. Specifically, we consider the problem of twisting lids of various bottle-like objects with two hands. This is the first sim-to-real RL system that enables such capabilities on bimanual multi-fingered hands.
  arXiv  Detail & Related papers  (2024-03-04T18:59:30Z)
• Controllable Human-Object Interaction Synthesis [77.56877961681462]
  We propose Controllable Human-Object Interaction Synthesis (CHOIS) to generate synchronized object motion and human motion in 3D scenes. Here, language descriptions inform style and intent, and waypoints, which can be effectively extracted from high-level planning, ground the motion in the scene. Our module seamlessly integrates with a path planning module, enabling the generation of long-term interactions in 3D environments.
  arXiv  Detail & Related papers  (2023-12-06T21:14:20Z)
• GRIP: Generating Interaction Poses Using Spatial Cues and Latent Consistency [57.9920824261925]
  Hands are dexterous and highly versatile manipulators that are central to how humans interact with objects and their environment. modeling realistic hand-object interactions is critical for applications in computer graphics, computer vision, and mixed reality. GRIP is a learning-based method that takes as input the 3D motion of the body and the object, and synthesizes realistic motion for both hands before, during, and after object interaction.
  arXiv  Detail & Related papers  (2023-08-22T17:59:51Z)
• Learning Explicit Contact for Implicit Reconstruction of Hand-held Objects from Monocular Images [59.49985837246644]
  We show how to model contacts in an explicit way to benefit the implicit reconstruction of hand-held objects. In the first part, we propose a new subtask of directly estimating 3D hand-object contacts from a single image. In the second part, we introduce a novel method to diffuse estimated contact states from the hand mesh surface to nearby 3D space.
  arXiv  Detail & Related papers  (2023-05-31T17:59:26Z)
• Physical Interaction: Reconstructing Hand-object Interactions with Physics [17.90852804328213]
  The paper proposes a physics-based method to better solve the ambiguities in the reconstruction. It first proposes a force-based dynamic model of the in-hand object, which recovers the unobserved contacts and also solves for plausible contact forces. Experiments show that the proposed technique reconstructs both physically plausible and more accurate hand-object interaction.
  arXiv  Detail & Related papers  (2022-09-22T07:41:31Z)
• Contact-Aware Retargeting of Skinned Motion [49.71236739408685]
  This paper introduces a motion estimation method that preserves self-contacts and prevents interpenetration. The method identifies self-contacts and ground contacts in the input motion, and optimize the motion to apply to the output skeleton. In experiments, our results quantitatively outperform previous methods and we conduct a user study where our retargeted motions are rated as higher-quality than those produced by recent works.
  arXiv  Detail & Related papers  (2021-09-15T17:05:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.