GIRAF: Towards Generalizable Human Interactions with Articulated Objects

• URL: http://arxiv.org/abs/2607.07880v1
• Date: Wed, 08 Jul 2026 19:30:44 GMT
• Title: GIRAF: Towards Generalizable Human Interactions with Articulated Objects
• Abstract summary: We present a text-conditioned diffusion model that addresses the challenge of Synthesizing realistic full-body human interactions with articulated objects.<n>Our method demonstrated strong generalization to unseen object configurations, surpassing current state-of-the-art methods.
• Score: 50.52795413484619
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Synthesizing realistic full-body human interactions with articulated objects is a fundamental challenge for embodied AI and graphics, with applications in robotics training and virtual agents. Existing models remain limited: some focus on simple activities with static objects, while others restrict attention to hand-only manipulation. This leaves open the problem of generating coordinated full-body motion that approaches, manipulates, and moves articulated objects in a realistic and generalizable way. The key difficulty lies in reasoning jointly about locomotion, fine-grained contact, and object articulation. Models must capture subtle hand-object correspondences that transfer across object geometries, while also producing seamless transitions from navigation to manipulation. At the same time, the scarcity of large-scale paired motion-scene data makes it difficult to generalize across diverse object positions and shapes. We introduce a text-conditioned diffusion model that addresses these challenges through three core ideas: an object-centric representation that unifies hand-object contact with object surfaces, a mixed-domain training strategy that balances locomotion and interaction, and a contact-based augmentation scheme that expands training diversity. Through experiments, our method demonstrated strong generalization to unseen object configurations, surpassing current state-of-the-art methods.

Related papers

• Unifying Physics-Based Humanoid Interaction with a Context-Conditioned Interaction Prior [20.58647951068921]
  CHIP is a physics-grounded framework for learning reusable humanoid interaction skills from heterogeneous motion data.<n>We first learn physics-based motion-imitation policies that acquire grounded teacher behaviors from heterogeneous interaction data.<n>We then distill these behaviors into a context-conditioned interaction prior that captures reusable motion structure across locomotion and manipulation.
  arXiv  Detail & Related papers  (2026-09-06T13:16:48Z)
• One Demonstration, Many Objects: Generalizing Manipulation via Local Contact Geometry [49.066054862620014]
  Learning from human demonstrations has emerged as a scalable alternative to robot teleoperation.<n>Recent sim-to-real RL methods incorporate such priors, but often omit rewards that explicitly incentivize precise contact.<n>We propose DemoMimic, a policy that manipulates objects by focusing on their geometry local to the contact points.
  arXiv  Detail & Related papers  (2026-09-01T23:04:35Z)
• LOME: Learning Human-Object Manipulation with Action-Conditioned Egocentric World Model [14.98308724969322]
  LOME can generate realistic human-object interactions as videos conditioned on an input image, a text prompt, and per-frame human actions.<n>LOME injects strong and precise action guidance into object manipulation by jointly estimating spatial human actions.<n>LOME paves the way for AR/VR experiences and scalable robotic training, without being limited to simulated environments.
  arXiv  Detail & Related papers  (2026-03-28T23:58:29Z)
• ByteLoom: Weaving Geometry-Consistent Human-Object Interactions through Progressive Curriculum Learning [19.292101162897975]
  We introduce ByteLoom, a framework that generates realistic HOI videos with geometrically consistent object illustration.<n>We first propose an RCM-cache mechanism that leverages Relative Coordinate Maps (RCM) as a universal representation to maintain object's geometry consistency.<n>We then design a training curriculum that enhances model capabilities in a progressive style and relaxes the demand of hand mesh.
  arXiv  Detail & Related papers  (2025-12-28T09:38:36Z)
• CoopDiff: Anticipating 3D Human-object Interactions via Contact-consistent Decoupled Diffusion [62.93198247045824]
  3D human-object interaction (HOI) anticipation aims to predict the future motion of humans and their manipulated objects, conditioned on the historical context.<n>We propose a novel contact-consistent decoupled diffusion framework CoopDiff, which employs two distinct branches to decouple human and object motion modeling.
  arXiv  Detail & Related papers  (2025-08-10T03:29:17Z)
• Diffgrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model [25.00532805042292]
  We propose a simple yet effective framework that jointly models the relationship between the body, hands, and the given object motion sequences.<n>We introduce novel contact-aware losses and incorporate a data-driven, carefully designed guidance.<n> Experimental results demonstrate that our approach outperforms the state-of-the-art method and generates plausible whole-body motion sequences.
  arXiv  Detail & Related papers  (2024-12-30T02:21:43Z)
• Controllable Human-Object Interaction Synthesis [77.56877961681462]
  We propose Controllable Human-Object Interaction Synthesis (CHOIS) to generate synchronized object motion and human motion in 3D scenes. Here, language descriptions inform style and intent, and waypoints, which can be effectively extracted from high-level planning, ground the motion in the scene. Our module seamlessly integrates with a path planning module, enabling the generation of long-term interactions in 3D environments.
  arXiv  Detail & Related papers  (2023-12-06T21:14:20Z)
• Object Motion Guided Human Motion Synthesis [22.08240141115053]
  We study the problem of full-body human motion synthesis for the manipulation of large-sized objects. We propose Object MOtion guided human MOtion synthesis (OMOMO), a conditional diffusion framework. We develop a novel system that captures full-body human manipulation motions by simply attaching a smartphone to the object being manipulated.
  arXiv  Detail & Related papers  (2023-09-28T08:22:00Z)
• Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations [61.659439423703155]
  TOHO: Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations. Our method generates continuous motions that are parameterized only by the temporal coordinate. This work takes a step further toward general human-scene interaction simulation.
  arXiv  Detail & Related papers  (2023-03-23T09:31:56Z)
• Learning to Transfer In-Hand Manipulations Using a Greedy Shape Curriculum [79.6027464700869]
  We show that natural and robust in-hand manipulation of simple objects in a dynamic simulation can be learned from a high quality motion capture example. We propose a simple greedy curriculum search algorithm that can successfully apply to a range of objects such as a teapot, bunny, bottle, train, and elephant.
  arXiv  Detail & Related papers  (2023-03-14T17:08:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.