GRACE: Estimating Geometry-level 3D Human-Scene Contact from 2D Images

• URL: http://arxiv.org/abs/2505.06575v1
• Date: Sat, 10 May 2025 09:25:46 GMT
• Title: GRACE: Estimating Geometry-level 3D Human-Scene Contact from 2D Images
• Authors: Chengfeng Wang, Wei Zhai, Yuhang Yang, Yang Cao, Zhengjun Zha,
• Abstract summary: Estimating the geometry level of human-scene contact aims to ground specific contact surface points at 3D human geometries.<n> GRACE (Geometry-level Reasoning for 3D Human-scene Contact Estimation) is a new paradigm for 3D human contact estimation.<n>It incorporates a point cloud encoder-decoder architecture along with a hierarchical feature extraction and fusion module.
• Score: 54.602947113980655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Estimating the geometry level of human-scene contact aims to ground specific contact surface points at 3D human geometries, which provides a spatial prior and bridges the interaction between human and scene, supporting applications such as human behavior analysis, embodied AI, and AR/VR. To complete the task, existing approaches predominantly rely on parametric human models (e.g., SMPL), which establish correspondences between images and contact regions through fixed SMPL vertex sequences. This actually completes the mapping from image features to an ordered sequence. However, this approach lacks consideration of geometry, limiting its generalizability in distinct human geometries. In this paper, we introduce GRACE (Geometry-level Reasoning for 3D Human-scene Contact Estimation), a new paradigm for 3D human contact estimation. GRACE incorporates a point cloud encoder-decoder architecture along with a hierarchical feature extraction and fusion module, enabling the effective integration of 3D human geometric structures with 2D interaction semantics derived from images. Guided by visual cues, GRACE establishes an implicit mapping from geometric features to the vertex space of the 3D human mesh, thereby achieving accurate modeling of contact regions. This design ensures high prediction accuracy and endows the framework with strong generalization capability across diverse human geometries. Extensive experiments on multiple benchmark datasets demonstrate that GRACE achieves state-of-the-art performance in contact estimation, with additional results further validating its robust generalization to unstructured human point clouds.

Related papers

• SeqAffordSplat: Scene-level Sequential Affordance Reasoning on 3D Gaussian Splatting [85.87902260102652]
  We introduce the novel task of Sequential 3D Gaussian Affordance Reasoning.<n>We then propose SeqSplatNet, an end-to-end framework that directly maps an instruction to a sequence of 3D affordance masks.<n>Our method sets a new state-of-the-art on our challenging benchmark, effectively advancing affordance reasoning from single-step interactions to complex, sequential tasks at the scene level.
  arXiv  Detail & Related papers  (2025-07-31T17:56:55Z)
• GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding [53.42728468191711]
  Open-Vocabulary 3D object affordance grounding aims to anticipate action possibilities'' regions on 3D objects with arbitrary instructions.<n>We propose GREAT (GeometRy-intEntion collAboraTive inference) for Open-Vocabulary 3D Object Affordance Grounding.
  arXiv  Detail & Related papers  (2024-11-29T11:23:15Z)
• StackFLOW: Monocular Human-Object Reconstruction by Stacked Normalizing Flow with Offset [56.71580976007712]
  We propose to use the Human-Object Offset between anchors which are densely sampled from the surface of human mesh and object mesh to represent human-object spatial relation. Based on this representation, we propose Stacked Normalizing Flow (StackFLOW) to infer the posterior distribution of human-object spatial relations from the image. During the optimization stage, we finetune the human body pose and object 6D pose by maximizing the likelihood of samples.
  arXiv  Detail & Related papers  (2024-07-30T04:57:21Z)
• Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation [32.30055363306321]
  We propose a paradigm for seamlessly unifying different human pose and shape-related tasks and datasets.<n>Our formulation is centered on the ability to query any arbitrary point of the human volume, and obtain its estimated location in 3D.
  arXiv  Detail & Related papers  (2024-07-10T10:44:18Z)
• Gaussian Control with Hierarchical Semantic Graphs in 3D Human Recovery [15.58274601909995]
  We introduce the Hierarchical Graph Human Gaussian Control (HUGS) framework for achieving high-fidelity 3D human reconstruction. Our approach involves leveraging explicitly semantic priors of body parts to ensure the consistency of geometric topology. Our method exhibits superior performance in human body reconstruction, particularly in enhancing surface details and accurately reconstructing body part junctions.
  arXiv  Detail & Related papers  (2024-05-21T03:40:56Z)
• PGAHum: Prior-Guided Geometry and Appearance Learning for High-Fidelity Animatable Human Reconstruction [9.231326291897817]
  We introduce PGAHum, a prior-guided geometry and appearance learning framework for high-fidelity animatable human reconstruction. We thoroughly exploit 3D human priors in three key modules of PGAHum to achieve high-quality geometry reconstruction with intricate details and photorealistic view synthesis on unseen poses.
  arXiv  Detail & Related papers  (2024-04-22T04:22:30Z)
• GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splatting [81.03553265684184]
  We introduce GeoGS3D, a framework for reconstructing detailed 3D objects from single-view images. We propose a novel metric, Gaussian Divergence Significance (GDS), to prune unnecessary operations during optimization. Experiments demonstrate that GeoGS3D generates images with high consistency across views and reconstructs high-quality 3D objects.
  arXiv  Detail & Related papers  (2024-03-15T12:24:36Z)
• CrossHuman: Learning Cross-Guidance from Multi-Frame Images for Human Reconstruction [6.450579406495884]
  CrossHuman is a novel method that learns cross-guidance from parametric human model and multi-frame RGB images. We design a reconstruction pipeline combined with tracking-based methods and tracking-free methods. Compared with previous works, our CrossHuman enables high-fidelity geometry details and texture in both visible and invisible regions.
  arXiv  Detail & Related papers  (2022-07-20T08:25:20Z)
• Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images [79.70127290464514]
  We decompose the task into two stages, i.e. person localization and pose estimation. And we propose three task-specific graph neural networks for effective message passing. Our approach achieves state-of-the-art performance on CMU Panoptic and Shelf datasets.
  arXiv  Detail & Related papers  (2021-09-13T11:44:07Z)
• HMOR: Hierarchical Multi-Person Ordinal Relations for Monocular Multi-Person 3D Pose Estimation [54.23770284299979]
  This paper introduces a novel form of supervision - Hierarchical Multi-person Ordinal Relations (HMOR) HMOR encodes interaction information as the ordinal relations of depths and angles hierarchically. An integrated top-down model is designed to leverage these ordinal relations in the learning process. The proposed method significantly outperforms state-of-the-art methods on publicly available multi-person 3D pose datasets.
  arXiv  Detail & Related papers  (2020-08-01T07:53:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.