Physics-Aware Human-Object Rendering from Sparse Views via 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2503.09640v1
• Date: Wed, 12 Mar 2025 04:19:21 GMT
• Title: Physics-Aware Human-Object Rendering from Sparse Views via 3D Gaussian Splatting
• Abstract summary: HOGS is a novel framework for efficient and physically plausible HOI rendering from sparse views.<n>It incorporates a Human Pose Refinement module and a Sparse-View Human-Object Contact Prediction module.<n>Experiments on the HODome dataset demonstrate that HOGS achieves superior rendering quality, efficiency, and physical plausibility compared to existing methods.
• Score: 41.369111968498814
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Rendering realistic human-object interactions (HOIs) from sparse-view inputs is challenging due to occlusions and incomplete observations, yet crucial for various real-world applications. Existing methods always struggle with either low rendering qualities (\eg, visual fidelity and physically plausible HOIs) or high computational costs. To address these limitations, we propose HOGS (Human-Object Rendering via 3D Gaussian Splatting), a novel framework for efficient and physically plausible HOI rendering from sparse views. Specifically, HOGS combines 3D Gaussian Splatting with a physics-aware optimization process. It incorporates a Human Pose Refinement module for accurate pose estimation and a Sparse-View Human-Object Contact Prediction module for efficient contact region identification. This combination enables coherent joint rendering of human and object Gaussians while enforcing physically plausible interactions. Extensive experiments on the HODome dataset demonstrate that HOGS achieves superior rendering quality, efficiency, and physical plausibility compared to existing methods. We further show its extensibility to hand-object grasp rendering tasks, presenting its broader applicability to articulated object interactions.

Related papers

• Generalizable Human Gaussian Splatting via Multi-view Semantic Consistency [6.489803560613637]
  We propose a novel method to accurately localize 3D Gaussians and improve the quality of human rendering.<n>The key idea is to unproject latent embeddings encoded from each viewpoint into a shared 3D space through predicted depth maps.<n> Experimental results on benchmark datasets show that the proposed method efficiently improves the performance of generalizable human Gaussian splatting from sparse-view inputs.
  arXiv  Detail & Related papers  (2026-04-28T10:12:28Z)
• Habitat-GS: A High-Fidelity Navigation Simulator with Dynamic Gaussian Splatting [83.60803397468139]
  Habitat-GS is a navigation-centric embodied AI simulator that integrates 3D photorealistic scene rendering and drivable gaussian avatars.<n>For dynamic human modeling, we introduce a gaussian avatar module that enables each avatar to simultaneously serve as a photorealistic visual entity and an effective navigation obstacle.
  arXiv  Detail & Related papers  (2026-04-14T11:52:59Z)
• HOIGS: Human-Object Interaction Gaussian Splatting [23.156838849980847]
  Reconstructing dynamic scenes with complex human-object interactions is a fundamental challenge in computer vision and graphics.<n>We propose Human-Object Interaction Gaussian Splatting (HOIGS), which explicitly models interaction-induced deformation between humans and objects.<n>Our method consistently outperforms state-of-the-art human-centric and 4D Gaussian approaches.
  arXiv  Detail & Related papers  (2026-04-05T08:27:28Z)
• Dynamic Avatar-Scene Rendering from Human-centric Context [75.95641456716373]
  We propose bf Separate-then-Map (StM) strategy to bridge separately defined and optimized models.<n>StM significantly outperforms existing state-of-the-art methods in both visual quality and rendering accuracy.
  arXiv  Detail & Related papers  (2025-11-13T17:39:06Z)
• HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Video [25.898073594115413]
  HoloScene is a novel interactive 3D reconstruction framework.<n>It encodes object geometry, appearance, and physical properties alongside hierarchical and inter-object relationships.<n>The resulting digital twins exhibit complete and precise geometry, physical stability, and realistic rendering from novel viewpoints.
  arXiv  Detail & Related papers  (2025-10-07T04:12:18Z)
• Reconstructing Close Human Interaction with Appearance and Proxemics Reasoning [50.76723760768117]
  Existing human pose estimation methods cannot recover plausible close interactions from in-the-wild videos.<n>We find that human appearance can provide a straightforward cue to address these obstacles.<n>We propose a dual-branch optimization framework to reconstruct accurate interactive motions with plausible body contacts constrained by human appearances, social proxemics, and physical laws.
  arXiv  Detail & Related papers  (2025-07-03T12:19:26Z)
• PIG: Physically-based Multi-Material Interaction with 3D Gaussians [14.097146027458368]
  PIG: Physically-Based Multi-Material Interaction with 3D Gaussians is a novel approach that combines 3D object segmentation with the simulation of objects interacting in high precision.<n>We show that our method not only outperforms the state-of-the-art (SOTA) in terms of visual quality, but also opens up new directions and pipelines for the field of physically realistic scene generation.
  arXiv  Detail & Related papers  (2025-06-09T11:25:21Z)
• Parametric Gaussian Human Model: Generalizable Prior for Efficient and Realistic Human Avatar Modeling [32.480049588166544]
  Photo and animatable human avatars are a key enabler for virtual/augmented reality, telepresence, and digital entertainment.<n>We present the Parametric Gaussian Human Model (PGHM), a generalizable and efficient framework that integrates human priors into 3DGS.<n>Experiments show that PGHM is significantly more efficient than optimization-from-scratch methods, requiring only approximately 20 minutes per subject to produce avatars with comparable visual quality.
  arXiv  Detail & Related papers  (2025-06-07T03:53:30Z)
• Zero-Shot Human-Object Interaction Synthesis with Multimodal Priors [31.277540988829976]
  This paper proposes a novel zero-shot HOI synthesis framework without relying on end-to-end training on currently limited 3D HOI datasets. We employ pre-trained human pose estimation models to extract human poses and introduce a generalizable category-level 6-DoF estimation method to obtain the object poses from 2D HOI images.
  arXiv  Detail & Related papers  (2025-03-25T23:55:47Z)
• StackFLOW: Monocular Human-Object Reconstruction by Stacked Normalizing Flow with Offset [56.71580976007712]
  We propose to use the Human-Object Offset between anchors which are densely sampled from the surface of human mesh and object mesh to represent human-object spatial relation. Based on this representation, we propose Stacked Normalizing Flow (StackFLOW) to infer the posterior distribution of human-object spatial relations from the image. During the optimization stage, we finetune the human body pose and object 6D pose by maximizing the likelihood of samples.
  arXiv  Detail & Related papers  (2024-07-30T04:57:21Z)
• Human-Aware 3D Scene Generation with Spatially-constrained Diffusion Models [16.259040755335885]
  Previous auto-regression-based 3D scene generation methods have struggled to accurately capture the joint distribution of multiple objects and input humans. We introduce two spatial collision guidance mechanisms: human-object collision avoidance and object-room boundary constraints. Our framework can generate more natural and plausible 3D scenes with precise human-scene interactions.
  arXiv  Detail & Related papers  (2024-06-26T08:18:39Z)
• HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splatting [53.6394928681237]
  holistic understanding of urban scenes based on RGB images is a challenging yet important problem. Our main idea involves the joint optimization of geometry, appearance, semantics, and motion using a combination of static and dynamic 3D Gaussians. Our approach offers the ability to render new viewpoints in real-time, yielding 2D and 3D semantic information with high accuracy.
  arXiv  Detail & Related papers  (2024-03-19T13:39:05Z)
• Diffusion Models are Efficient Data Generators for Human Mesh Recovery [55.37787289869703]
  We show that synthetic data created by generative models is complementary to CG-rendered data.<n>We propose an effective data generation pipeline based on recent diffusion models, termed HumanWild.<n>Our work could pave the way for scaling up 3D human recovery to in-the-wild scenes.
  arXiv  Detail & Related papers  (2024-03-17T06:31:16Z)
• Scaling Up Dynamic Human-Scene Interaction Modeling [58.032368564071895]
  TRUMANS is the most comprehensive motion-captured HSI dataset currently available. It intricately captures whole-body human motions and part-level object dynamics. We devise a diffusion-based autoregressive model that efficiently generates HSI sequences of any length.
  arXiv  Detail & Related papers  (2024-03-13T15:45:04Z)
• Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Models [8.933560282929726]
  We introduce a novel affordance representation, named Comprehensive Affordance (ComA) Given a 3D object mesh, ComA models the distribution of relative orientation and proximity of vertices in interacting human meshes. We demonstrate that ComA outperforms competitors that rely on human annotations in modeling contact-based affordance.
  arXiv  Detail & Related papers  (2024-01-23T18:59:59Z)
• HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting [113.37908093915837]
  Existing methods optimize 3D representations like mesh or neural fields via score distillation sampling (SDS), which suffers from inadequate fine details or excessive training time. In this paper, we propose an efficient yet effective framework, HumanGaussian, that generates high-quality 3D humans with fine-grained geometry and realistic appearance.
  arXiv  Detail & Related papers  (2023-11-28T18:59:58Z)
• NeuralFusion: Neural Volumetric Rendering under Human-object Interactions [46.70371238621842]
  We propose a neural approach for volumetric human-object capture and rendering using sparse consumer RGBD sensors. For geometry modeling, we propose a neural implicit inference scheme with non-rigid key-volume fusion. We also introduce a layer-wise human-object texture rendering scheme, which combines volumetric and image-based rendering in both spatial and temporal domains.
  arXiv  Detail & Related papers  (2022-02-25T17:10:07Z)
• HSPACE: Synthetic Parametric Humans Animated in Complex Environments [67.8628917474705]
  We build a large-scale photo-realistic dataset, Human-SPACE, of animated humans placed in complex indoor and outdoor environments. We combine a hundred diverse individuals of varying ages, gender, proportions, and ethnicity, with hundreds of motions and scenes, in order to generate an initial dataset of over 1 million frames. Assets are generated automatically, at scale, and are compatible with existing real time rendering and game engines.
  arXiv  Detail & Related papers  (2021-12-23T22:27:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.