Surface Keypoint Representation for Multi-Object and Articulated Human-Object Interaction Generation

• URL: http://arxiv.org/abs/2608.03158v1
• Date: Tue, 04 Aug 2026 05:47:07 GMT
• Title: Surface Keypoint Representation for Multi-Object and Articulated Human-Object Interaction Generation
• Abstract summary: We propose surfacepoint trajectories as an object motion representation for each rigid component.<n>This representation handles multi-object coordination and diverse articulation mechanisms directly from point dynamics.<n>We factorize HOI generation into three stages: generating object motions from text or waypoints, predicting the contact distance field, and synthesizing the contact distance field.
• Score: 45.67585781847749
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Daily activities require humans to coordinate whole-body motion with the motion of surrounding objects. Despite recent progress in human-object interaction (HOI) generation, most existing methods assume interactions with a single rigid object and do not extend well to scenarios involving a variable number of objects or articulated objects with diverse joint mechanisms. We propose surface keypoint trajectories as an object motion representation: for each rigid component, whether a standalone object or one part of an articulated assembly, we track a small set of non-collinear surface points over time. This representation handles multi-object coordination and diverse articulation mechanisms directly from point dynamics without requiring explicit joint-type specification. To model when and where each body region contacts each object, we introduce a spatio-temporal contact distance field that extends distance-based contact modeling to whole-body, multi-object, and articulated settings. We factorize HOI generation into three stages: generating object motions from text or waypoints, predicting the contact distance field, and synthesizing whole-body motion with contact-guided optimization. Experiments on ParaHome, HIMO, ARCTIC, and OMOMO demonstrate better or comparable performance to existing methods across single-object, multi-object, and articulated interaction settings.

Related papers

• GIRAF: Towards Generalizable Human Interactions with Articulated Objects [50.52795413484619]
  We present a text-conditioned diffusion model that addresses the challenge of Synthesizing realistic full-body human interactions with articulated objects.<n>Our method demonstrated strong generalization to unseen object configurations, surpassing current state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-07-08T19:30:44Z)
• MMHOI: Modeling Complex 3D Multi-Human Multi-Object Interactions [20.96140289964853]
  We present MMHOI -- a large-scale, Multi-human Multi-object Interaction dataset consisting of images from 12 everyday scenarios.<n> MMHOI offers complete 3D shape and pose annotations for every person and object, along with labels for 78 action categories and 14 interaction-specific body parts.<n>We present MMHOI-Net, an end-to-end transformer-based neural network for jointly estimating human-object 3D geometries, their interactions, and associated actions.
  arXiv  Detail & Related papers  (2025-10-09T06:18:12Z)
• InteractMove: Text-Controlled Human-Object Interaction Generation in 3D Scenes with Movable Objects [15.92165183796286]
  We propose a novel task of text-controlled human object interaction generation in 3D scenes with movable objects.<n>Existing human-scene interaction datasets suffer from insufficient interaction categories.<n>We propose a hand-object joint affordance learning to predict contact regions for different hand joints.
  arXiv  Detail & Related papers  (2025-09-28T03:29:15Z)
• CoopDiff: Anticipating 3D Human-object Interactions via Contact-consistent Decoupled Diffusion [62.93198247045824]
  3D human-object interaction (HOI) anticipation aims to predict the future motion of humans and their manipulated objects, conditioned on the historical context.<n>We propose a novel contact-consistent decoupled diffusion framework CoopDiff, which employs two distinct branches to decouple human and object motion modeling.
  arXiv  Detail & Related papers  (2025-08-10T03:29:17Z)
• EigenActor: Variant Body-Object Interaction Generation Evolved from Invariant Action Basis Reasoning [66.68366281305977]
  This paper explores a cross-modality synthesis task that infers 3D human-object interactions (HOIs) from a given text-based instruction.<n>Existing text-to-HOI synthesis methods mainly deploy a direct mapping from texts to object-specific 3D body motions.<n>We propose a novel body pose generation strategy for the text-to-HOI task: infer object-agnostic canonical body action first and then enrich object-specific interaction styles.
  arXiv  Detail & Related papers  (2025-03-01T07:15:10Z)
• Controllable Human-Object Interaction Synthesis [77.56877961681462]
  We propose Controllable Human-Object Interaction Synthesis (CHOIS) to generate synchronized object motion and human motion in 3D scenes. Here, language descriptions inform style and intent, and waypoints, which can be effectively extracted from high-level planning, ground the motion in the scene. Our module seamlessly integrates with a path planning module, enabling the generation of long-term interactions in 3D environments.
  arXiv  Detail & Related papers  (2023-12-06T21:14:20Z)
• Full-Body Articulated Human-Object Interaction [61.01135739641217]
  CHAIRS is a large-scale motion-captured f-AHOI dataset consisting of 16.2 hours of versatile interactions. CHAIRS provides 3D meshes of both humans and articulated objects during the entire interactive process. By learning the geometrical relationships in HOI, we devise the very first model that leverage human pose estimation.
  arXiv  Detail & Related papers  (2022-12-20T19:50:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.