OneHOI: Unifying Human-Object Interaction Generation and Editing

• URL: http://arxiv.org/abs/2604.14062v1
• Date: Wed, 15 Apr 2026 16:37:36 GMT
• Title: OneHOI: Unifying Human-Object Interaction Generation and Editing
• Abstract summary: OneHOI is a unified diffusion transformer framework that consolidates HOI generation and editing into a single conditional denoising process.<n>OneHOI supports layout-guided, layout-free, arbitrary-mask, and mixed-condition control, achieving state-of-the-art results across both HOI generation and editing.
• Score: 36.931038095565235
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human-Object Interaction (HOI) modelling captures how humans act upon and relate to objects, typically expressed as <person, action, object> triplets. Existing approaches split into two disjoint families: HOI generation synthesises scenes from structured triplets and layout, but fails to integrate mixed conditions like HOI and object-only entities; and HOI editing modifies interactions via text, yet struggles to decouple pose from physical contact and scale to multiple interactions. We introduce OneHOI, a unified diffusion transformer framework that consolidates HOI generation and editing into a single conditional denoising process driven by shared structured interaction representations. At its core, the Relational Diffusion Transformer (R-DiT) models verb-mediated relations through role- and instance-aware HOI tokens, layout-based spatial Action Grounding, a Structured HOI Attention to enforce interaction topology, and HOI RoPE to disentangle multi-HOI scenes. Trained jointly with modality dropout on our HOI-Edit-44K, along with HOI and object-centric datasets, OneHOI supports layout-guided, layout-free, arbitrary-mask, and mixed-condition control, achieving state-of-the-art results across both HOI generation and editing. Code is available at https://jiuntian.github.io/OneHOI/.

Related papers

• Taming I2V models for Image HOI Editing: A Cognitive Benchmark and Agentic Self-Correcting Framework [62.705866028818576]
  Current image editing methods excel at static attributes but fail at complex Human-Object Interactions (HOI)<n>We introduce HOI-Edit, a comprehensive benchmark with three progressive cognitive levels.<n>On HOI-Edit, SCPE achieves performance competitive with state-of-the-art (SOTA) editing models like Nano Banana on interaction.
  arXiv  Detail & Related papers  (2026-06-17T13:44:26Z)
• InterEdit: Navigating Text-Guided Multi-Human 3D Motion Editing [73.51964472028392]
  We introduce the task of multi-person 3D motion editing, where a target motion is generated from a source and a text instruction.<n>To support this, we propose InterEdit3D, a new dataset with manual two-person motion change annotations, and a Text-guided Multi-human Motion Editing (TMME) benchmark.<n>InterEdit improves text-to-motion consistency and edit fidelity, achieving state-of-the-art TMME performance.
  arXiv  Detail & Related papers  (2026-03-13T15:30:51Z)
• Interaction-aware Representation Modeling with Co-occurrence Consistency for Egocentric Hand-Object Parsing [20.40288070674112]
  We propose an end-to-end Interaction-aware Transformer (InterFormer)<n>It integrates three key components, i.e., a Dynamic Query Generator (DQG), a Dual-context Feature Selector (DFS), and the Conditional Co-occurrence (CoCo) loss.<n>Our model achieves state-of-the-art performance on both the EgoHOS and the challenging out-of-distribution mini-HOI4D datasets.
  arXiv  Detail & Related papers  (2026-02-24T06:39:18Z)
• Learning to Generate Human-Human-Object Interactions from Textual Descriptions [15.38195247862565]
  We present a novel research problem to model the correlations between two people engaged in a shared interaction involving an object.<n>We refer to this formulation as Human-Human-Object Interactions (HHOIs)<n>We present a newly captured HHOIs dataset and a method to synthesize HHOI data by leveraging image generative models.
  arXiv  Detail & Related papers  (2025-11-25T16:17:23Z)
• HOSIG: Full-Body Human-Object-Scene Interaction Generation with Hierarchical Scene Perception [57.37135310143126]
  HO SIG is a novel framework for synthesizing full-body interactions through hierarchical scene perception.<n>Our framework supports unlimited motion length through autoregressive generation and requires minimal manual intervention.<n>This work bridges the critical gap between scene-aware navigation and dexterous object manipulation.
  arXiv  Detail & Related papers  (2025-06-02T12:08:08Z)
• InteractEdit: Zero-Shot Editing of Human-Object Interactions in Images [42.8116807595149]
  We present InteractEdit, a novel framework for zero-shot Human-Object Interaction (HOI) editing.<n>It transforms an existing interaction in an image into a new, desired interaction while preserving the identities of the subject and object.<n>Our experiments show that InteractEdit significantly outperforms existing methods.
  arXiv  Detail & Related papers  (2025-03-12T07:40:45Z)
• ObjectRelator: Enabling Cross-View Object Relation Understanding Across Ego-Centric and Exo-Centric Perspectives [109.11714588441511]
  The Ego-Exo object correspondence task aims to understand object relations across ego-exo perspectives through segmentation.<n> PSALM, a recently proposed segmentation method, stands out as a notable exception with its demonstrated zero-shot ability on this task.<n>We propose ObjectRelator, a novel approach featuring two key modules: Multimodal Condition Fusion and SSL-based Cross-View Object Alignment.
  arXiv  Detail & Related papers  (2024-11-28T12:01:03Z)
• HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects [86.86284624825356]
  HIMO is a dataset of full-body human interacting with multiple objects. HIMO contains 3.3K 4D HOI sequences and 4.08M 3D HOI frames.
  arXiv  Detail & Related papers  (2024-07-17T07:47:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.