JointHOI: Jointly Generating Contact Maps Enhances Hand Object Interaction Generation

• URL: http://arxiv.org/abs/2607.01768v1
• Date: Thu, 02 Jul 2026 06:34:37 GMT
• Title: JointHOI: Jointly Generating Contact Maps Enhances Hand Object Interaction Generation
• Abstract summary: We present JointHOI, a single stage diffusion framework that jointly generates 3D hand object motion and dynamic, distance based contact maps from text.<n> Experiments on GRAB and ARCTIC demonstrate consistent improvements in text adherence and physical plausibility over prior methods.
• Score: 24.655154886327633
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text driven hand object interaction (HOI) generation is gaining attention for immersive applications and robotics, yet producing physically plausible interactions remains challenging. Even when individual motions appear natural, small contact errors can cause conspicuous artifacts such as floating and interpenetration. Prior methods mitigate these issues using explicit contact cues or implicit grasp priors, but typically rely on multi stage pipelines and fail to model temporally evolving contact. We present JointHOI, a single stage diffusion framework that jointly generates 3D hand object motion and dynamic, distance based contact maps from text. By treating contact as an auxiliary inner modality, joint generation enables the model to learn contact motion coupling during training. At inference, contact guided sampling enforces consistency between generated contact maps and motion implied geometry, improving temporal stability and reducing penetration and floating. Experiments on GRAB and ARCTIC demonstrate consistent improvements in text adherence and physical plausibility over prior methods.

Related papers

• GIRAF: Towards Generalizable Human Interactions with Articulated Objects [50.52795413484619]
  We present a text-conditioned diffusion model that addresses the challenge of Synthesizing realistic full-body human interactions with articulated objects.<n>Our method demonstrated strong generalization to unseen object configurations, surpassing current state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-07-08T19:30:44Z)
• Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlation [74.87716678195099]
  Visuo-Tactile policies leveraging optical tactile sensors have shown great promise in contact-rich manipulation.<n>Despite their advantages, extracting fine-grained contact states necessary for contact-rich manipulation remains an open challenge.<n>We propose a motion-aware tactile representation to facilitate contact-rich manipulation.
  arXiv  Detail & Related papers  (2026-06-29T08:20:18Z)
• DragMesh-2: Physically Plausible Dexterous Hand-Object Interaction with Articulated Objects [12.35353010124029]
  Dexterous interaction with articulated objects is important for household, assistive, and humanoid manipulation.<n>We present DragMesh-2, a contact-driven framework for dexterous interaction with articulated objects.<n>We also propose PICA, a physically informed contact-aware training mechanism.
  arXiv  Detail & Related papers  (2026-06-13T05:47:14Z)
• Hand-centric Human-to-Robot Trajectory Transfer from Video Demonstrations via Open-World Contact Localization [28.23926683554352]
  We present emphHOWTransfer, a hand-centric framework that distills human demonstrations into contact-aware, taxonomy-informed, and diverse robotic trajectories.<n>emphHOWTransfer recovers temporally consistent 3D hand motion and localizes temporal contact intervals by reasoning over observed hand-object interaction cues.<n>Experiments show that emphHOWTransfer enables accurate contact localization and high-quality robot motion with $86%$ success.
  arXiv  Detail & Related papers  (2026-06-09T11:53:29Z)
• CHOIR: Contact-aware 4D Hand-Object Interaction Reconstruction [73.97421249653412]
  We present CHOIR, a Contact-aware HOI Reconstruction framework for a monocular camera.<n>ChoIR first initializes a coarse, contact-agnostic 4D HOI sequence from open-world visual priors.<n>It then introduces a generative HOI spatial rectification module to predict ray-depth corrections and rectify hand-object relative placement.<n>Last, a contact-aware joint optimization with dynamically updated contact constraints enforces geometric, temporal, and contact consistency.
  arXiv  Detail & Related papers  (2026-05-20T10:31:10Z)
• SocialMirror: Reconstructing 3D Human Interaction Behaviors from Monocular Videos with Semantic and Geometric Guidance [49.69016078147708]
  Accurately reconstructing human behavior in close-interaction scenarios is crucial for enabling realistic virtual interactions in augmented reality.<n>We propose SocialMirror, a diffusion-based framework that integrates semantic and geometric cues to effectively address these issues.<n>SocialMirror achieves state-of-the-art performance in reconstructing interactive human meshes, demonstrating strong generalization across unseen datasets and in-the-wild scenarios.
  arXiv  Detail & Related papers  (2026-04-15T07:41:52Z)
• Unleashing Guidance Without Classifiers for Human-Object Interaction Animation [54.24065561632133]
  Prior diffusion-based approaches often rely on hand-crafted contact priors or human-imposed kinematic constraints to improve contact quality.<n>We propose a data-driven alternative in which guidance emerges from the denoising pace itself, reducing dependence on manually designed priors.<n>We find that this data-driven guidance is inherently contact-aware, and can be enhanced when training is augmented with a broad spectrum of synthetic object geometries.
  arXiv  Detail & Related papers  (2026-03-26T17:59:34Z)
• Towards Realistic Hand-Object Interaction with Gravity-Field Based Diffusion Bridge [23.484478357333856]
  We formulate hand-object interaction as an attraction-driven process and propose a Gravity-Field Based Diffusion Bridge (GravityDB) to simulate interactions between a deformable hand surface and rigid objects.<n>Our approach effectively resolves the aforementioned issues by generating physically plausible interactions that are free of interpenetration, ensure stable grasping, and capture realistic hand deformations.
  arXiv  Detail & Related papers  (2025-09-03T08:15:19Z)
• Contact2Grasp: 3D Grasp Synthesis via Hand-Object Contact Constraint [18.201389966034263]
  3D grasp synthesis generates grasping poses given an input object. We introduce an intermediate variable for grasp contact areas to constrain the grasp generation. Our method outperforms state-of-the-art methods regarding grasp generation on various metrics.
  arXiv  Detail & Related papers  (2022-10-17T16:39:25Z)
• Contact-Aware Retargeting of Skinned Motion [49.71236739408685]
  This paper introduces a motion estimation method that preserves self-contacts and prevents interpenetration. The method identifies self-contacts and ground contacts in the input motion, and optimize the motion to apply to the output skeleton. In experiments, our results quantitatively outperform previous methods and we conduct a user study where our retargeted motions are rated as higher-quality than those produced by recent works.
  arXiv  Detail & Related papers  (2021-09-15T17:05:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.