HOIMask: Towards Generative Masked Modeling for Human Object Interaction Generation

• URL: http://arxiv.org/abs/2608.15141v1
• Date: Sat, 15 Aug 2026 09:32:17 GMT
• Title: HOIMask: Towards Generative Masked Modeling for Human Object Interaction Generation
• Abstract summary: HOIMask is the first generative masked framework for modeling HOI motion in discrete space.<n>It encodes both motion sequences and contact-aware signals into discrete 2D human and object token maps.<n>We introduce a novel contact-aware reconstruction guidance in discrete space during inference, which fuses contact signals to optimize HOI tokens.
• Score: 9.000468884843764
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion-based methods have dominated the HOI generation, as they enable critical contact fusions or signals to guide the diffusion process. However, they often result in high artifacts and unstable interaction quality due to error accumulation during iterative denoising. In this work, we propose HOIMask, the first generative masked framework for modeling HOI motion in discrete space. HOIMask first encodes both motion sequences and contact-aware signals into discrete 2D human and object token maps via HOI Vector Quantization (VQ), preserving fine-grained spatial-temporal structure beyond conventional 1D representations. On this basis, a generative masked modeling framework is employed to jointly capture human-object interaction dynamics, leveraging a transformer architecture designed to model complex spatial-temporal and interaction dependencies. To generate more coherent and physically plausible motions, we further introduce a novel contact-aware reconstruction guidance in discrete space during inference, which fuses contact signals to optimize HOI tokens that forces the generated motion with higher spatio-temporal consistency. With craftily designed motion interaction tokens, dedicated architecture and guidance strategy, HOIMask outperforms state-of-the-art diffusion-based methods, generating more realistic and semantically aligned HOI motions. Please refer to https://jyhflash.github.io/HOIMask/ for more results.

Related papers

• Spatiotemporally Decoupled Autoregressive Diffusion Model for Human Motion Generation [19.887078319028102]
  We propose a unifiedtemporal decoupled framework named DeMoDiff to redesign motion representation and architecture.<n>Our framework achieves state-of-the-art reconstruction performance and compelling motion generation results.
  arXiv  Detail & Related papers  (2026-08-24T14:04:35Z)
• DyG$^2$T: Modeling Object Dynamics with 3D Gaussian Temporal-Spatial Particle Graph Transformer [86.4618122245946]
  DyG$2$T is a dynamics modeling framework that infers object motion trajectories by spatially completing and temporally discriminating Key Point representations.<n>Experiments on both synthetic and real-world datasets demonstrate that DyG$2$T achieves accurate dynamics modeling and reasoning.
  arXiv  Detail & Related papers  (2026-08-19T03:44:28Z)
• STAG: Spatio-temporal Evolving Structural Representation of Action Units for Micro-expression Recognition [0.7799711162530713]
  Micro-expression recognition is challenging due to subtle and short-lived facial muscle movements.<n>Existing methods rely heavily on apex-onset frames, overlook fine-grained inter-frame dynamics, and separately model spatial and temporal information.<n>We propose STAG, a dynamic ROI-AU-coupled spatial-temporal network that jointly models motion flow and adaptive facial connectivity.
  arXiv  Detail & Related papers  (2026-06-26T13:46:48Z)
• HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching [113.81911881001905]
  HO-Flow is a framework for synthesizing realistic hand-object motion sequences from texts and canoncial 3D objects.<n>It first employs an interaction-aware variational autoencoder to encode sequences of hand and object motions into a unified latent manifold.<n>It then leverages a masked flow matching model that combines auto-regressive temporal reasoning with continuous latent generation.
  arXiv  Detail & Related papers  (2026-04-12T22:06:11Z)
• TriC-Motion: Tri-Domain Causal Modeling Grounded Text-to-Motion Generation [61.94780858309546]
  Tri-Domain Causal Text-to-Motion Generation (TriC-Motion) is a novel diffusion-based framework integrating spatial-domain modeling with causal intervention.<n>TriC-Motion achieves superior performance compared to state-of-the-art methods, attaining an outstanding R@1 of 0.612 on the HumanML3D dataset.
  arXiv  Detail & Related papers  (2026-02-09T10:12:13Z)
• SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios [48.09735396455107]
  Hand-Object Interaction (HOI) generation has significant application potential.<n>Current 3D HOI motion generation approaches heavily rely on predefined 3D object models and lab-captured motion data.<n>We propose a novel framework that combines visual priors and dynamic constraints within a synchronized diffusion process to generate the HOI video and motion simultaneously.
  arXiv  Detail & Related papers  (2025-06-03T05:04:29Z)
• GestureLSM: Latent Shortcut based Co-Speech Gesture Generation with Spatial-Temporal Modeling [32.47567372398872]
  GestureLSM is a flow-matching-based approach for Co-Speech Gesture Generation with spatial-temporal modeling.<n>It achieves state-of-the-art performance on BEAT2 while significantly reducing inference time compared to existing methods.
  arXiv  Detail & Related papers  (2025-01-31T05:34:59Z)
• Spatio-Temporal Branching for Motion Prediction using Motion Increments [55.68088298632865]
  Human motion prediction (HMP) has emerged as a popular research topic due to its diverse applications. Traditional methods rely on hand-crafted features and machine learning techniques. We propose a noveltemporal-temporal branching network using incremental information for HMP.
  arXiv  Detail & Related papers  (2023-08-02T12:04:28Z)
• GraMMaR: Ground-aware Motion Model for 3D Human Motion Reconstruction [61.833152949826946]
  We propose a novel Ground-aware Motion Model for 3D Human Motion Reconstruction, named GraMMaR. GraMMaR learns the distribution of transitions in both pose and interaction between every joint and ground plane at each time step of a motion sequence. It is trained to explicitly promote consistency between the motion and distance change towards the ground.
  arXiv  Detail & Related papers  (2023-06-29T07:22:20Z)
• Guided Motion Diffusion for Controllable Human Motion Synthesis [18.660523853430497]
  We propose Guided Motion Diffusion (GMD), a method that incorporates spatial constraints into the motion generation process. Specifically, we propose an effective feature projection scheme that manipulates motion representation to enhance the coherency between spatial information and local poses. Our experiments justify the development of GMD, which achieves a significant improvement over state-of-the-art methods in text-based motion generation.
  arXiv  Detail & Related papers  (2023-05-21T21:54:31Z)
• A Spatial-Temporal Attentive Network with Spatial Continuity for Trajectory Prediction [74.00750936752418]
  We propose a novel model named spatial-temporal attentive network with spatial continuity (STAN-SC) First, spatial-temporal attention mechanism is presented to explore the most useful and important information. Second, we conduct a joint feature sequence based on the sequence and instant state information to make the generative trajectories keep spatial continuity.
  arXiv  Detail & Related papers  (2020-03-13T04:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.