GenHOI: Towards Object-Consistent Hand-Object Interaction with Temporally Balanced and Spatially Selective Object Injection

• URL: http://arxiv.org/abs/2603.06048v1
• Date: Fri, 06 Mar 2026 09:01:09 GMT
• Title: GenHOI: Towards Object-Consistent Hand-Object Interaction with Temporally Balanced and Spatially Selective Object Injection
• Abstract summary: GenHOI is a lightweight augmentation to pretrained video generation models.<n>It injects reference-object information in a temporally balanced and spatially selective manner.<n>GenHOI significantly outperforms state-of-the-art HOI reenactment and all-in-one video editing methods.
• Score: 54.879037588415656
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hand-Object Interaction (HOI) remains a core challenge in digital human video synthesis, where models must generate physically plausible contact and preserve object identity across frames. Although recent HOI reenactment approaches have achieved progress, they are typically trained and evaluated in-domain and fail to generalize to complex, in-the-wild scenarios. In contrast, all-in-one video editing models exhibit broader robustness but still struggle with HOI-specific issues such as inconsistent object appearance. In this paper, we present GenHOI, a lightweight augmentation to pretrained video generation models that injects reference-object information in a temporally balanced and spatially selective manner. For temporal balancing, we propose Head-Sliding RoPE, which assigns head-specific temporal offsets to reference tokens, distributing their influence evenly across frames and mitigating the temporal decay of 3D RoPE to improve long-range object consistency. For spatial selectivity, we design a two-level spatial attention gate that concentrates object-conditioned attention on HOI regions and adaptively scales its strength, preserving background realism while enhancing interaction fidelity. Extensive qualitative and quantitative evaluations on unseen, in-the-wild scenes demonstrate that GenHOI significantly outperforms state-of-the-art HOI reenactment and all-in-one video editing methods. Project page: https://xuanhuang0.github.io/GenHOI/

Related papers

• DyG$^2$T: Modeling Object Dynamics with 3D Gaussian Temporal-Spatial Particle Graph Transformer [86.4618122245946]
  DyG$2$T is a dynamics modeling framework that infers object motion trajectories by spatially completing and temporally discriminating Key Point representations.<n>Experiments on both synthetic and real-world datasets demonstrate that DyG$2$T achieves accurate dynamics modeling and reasoning.
  arXiv  Detail & Related papers  (2026-08-19T03:44:28Z)
• SparseCtrl-HOI: Sparse Temporal Control for Human-Object Interaction Video Generation [29.683204938136758]
  Human-Object Interaction generation aims to synthesize realistic humans manipulating diverse objects.<n>Existing approaches to this problem rely on dense temporal guidance.<n>We introduce SparseCtrl-HOI, a novel sparse temporal control framework for HOI video generation.
  arXiv  Detail & Related papers  (2026-07-07T08:29:25Z)
• Tempered Self-Similarity Alignment for Physically Plausible Video Generation [83.40337664171939]
  Self-similarity loss transforms into probabilistic correspondence and trains the video generative model to align its correspondence with those of the visual foundation model on dynamically changing regions.<n>Our method demonstrates substantial improvements in physical plausibility across diverse interaction scenarios, validating the effectiveness of transferring relational knowledge for video generation.
  arXiv  Detail & Related papers  (2026-05-24T09:28:05Z)
• InfBaGel: Human-Object-Scene Interaction Generation with Dynamic Perception and Iterative Refinement [15.184022425476256]
  Human-object-scene interactions (HOSI) generation has broad applications in embodied AI, simulation, and animation.<n>We propose a coarse-to-fine instruction-conditioned interaction generation framework that is aligned with the iterative denoising process of a consistency model.<n>Our method achieves state-of-the-art performance in both HOSI and HOI generation, and strong to unseen scenes.
  arXiv  Detail & Related papers  (2026-04-06T16:44:02Z)
• RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space [51.441415833480505]
  RAYNOVA is a multiview world model for driving scenarios that employs a dual-causal autoregressive framework.<n>It constructs an isotropic-temporal representation across views, frames, and scales based on relative Plcker-ray positional encoding.
  arXiv  Detail & Related papers  (2026-02-24T08:41:40Z)
• AGILE: Hand-Object Interaction Reconstruction from Video via Agentic Generation [45.753757870577196]
  We introduce AGILE, a robust framework that shifts the paradigm from reconstruction to agentic generation for interaction learning.<n>We show that AGILE outperforms baselines in global geometric accuracy while demonstrating exceptional robustness on challenging sequences where prior art frequently collapses.
  arXiv  Detail & Related papers  (2026-02-04T15:42:58Z)
• iDiT-HOI: Inpainting-based Hand Object Interaction Reenactment via Video Diffusion Transformer [43.58952721477297]
  This paper presents a novel framework iDiT-HOI that enables in-the-wild HOI reenactment generation.<n> Specifically, we propose a unified inpainting-based token process method, called Inp-TPU, with a two-stage video diffusion transformer (DiT) model.
  arXiv  Detail & Related papers  (2025-06-15T13:41:43Z)
• Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Model [72.90370736032115]
  We present a novel video Reenactment framework focusing on Human-Object Interaction (HOI) via an adaptive layout-instructed Diffusion model (Re-HOLD)<n>Our key insight is to employ specialized layout representation for hands and objects, respectively.<n>To further improve the generation quality of HOI, we design an interactive textural enhancement module for both hands and objects.
  arXiv  Detail & Related papers  (2025-03-21T08:40:35Z)
• ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptors [73.26004792375556]
  This paper shows that robustness and generalisation to novel scene objects in 3D object-aware character synthesis can be achieved by training a motion model with as few as one reference object. We leverage an implicit feature representation trained on object-only datasets, which encodes an SE(3)-equivariant descriptor field around the object. We demonstrate substantial improvements in 3D virtual character motion and interaction quality and robustness to scenarios with unseen objects.
  arXiv  Detail & Related papers  (2023-08-24T17:59:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.