HAFMat: Hybrid Priors Guided Adaptive Fusion for Single-Image Human Material Estimation

• URL: http://arxiv.org/abs/2606.16323v1
• Date: Mon, 15 Jun 2026 07:30:09 GMT
• Title: HAFMat: Hybrid Priors Guided Adaptive Fusion for Single-Image Human Material Estimation
• Abstract summary: We propose HAFMat, a hybrid-prior-guided framework for single-image human material estimation.<n>Our method introduces guidance maps that encode complementary cues, including appearance, body geometry, structure, and prior material predictions from pre-trained models.<n>To exploit this property, we design a Multi-layer Adaptive Feature Fusion Mechanism, which adaptively fuses guidance features with decoder features at different stages.
• Score: 29.582037739752735
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physically based rendering (PBR) material estimation is a fundamental appearance decomposition task with broad applications in virtual content creation, relighting, and digital human rendering. However, estimating PBR materials from a single human image remains highly ill-posed, since illumination, geometry, and reflectance are heavily entangled in the observed appearance. To mitigate this ambiguity, we propose HAFMat, a hybrid-prior-guided framework for single-image human material estimation. Our method introduces guidance maps that encode complementary cues, including appearance, body geometry, structure, and prior material predictions from pre-trained models. A key observation is that these guidance cues are heterogeneous: some cues mainly provide texture-level constraints, while others convey higher-level semantic information. To exploit this property, we design a Multi-layer Adaptive Feature Fusion Mechanism, which adaptively fuses guidance features with decoder features at different stages. This design enables texture-dominant and semantic-dominant cues to guide material decoding at appropriate levels, leading to more accurate and physically plausible material estimation. Extensive experiments on both synthetic and real data demonstrate that our method achieves state-of-the-art performance in material estimation and downstream relighting.

Related papers

• Image2Garment: Simulation-ready Garment Generation from a Single Image [52.37273643091814]
  We propose a vision-language model to infer material composition and fabric attributes from real images.<n>We then train a lightweight predictor that maps these attributes to the corresponding physical fabric parameters.<n> Experiments show that our estimator achieves superior accuracy in material composition estimation and fabric attribute prediction.
  arXiv  Detail & Related papers  (2026-01-14T17:47:33Z)
• MatE: Material Extraction from Single-Image via Geometric Prior [36.8533172704247]
  MatE is a novel method for generating tileable PBR materials from a single image taken under unconstrained, real-world conditions.<n>We demonstrate the efficacy and robustness of our approach, enabling users to create realistic materials from real-world image.
  arXiv  Detail & Related papers  (2025-12-20T10:53:49Z)
• Intrinsic Image Fusion for Multi-View 3D Material Reconstruction [49.43509537480623]
  We introduce Intrinsic Image Fusion, a method that reconstructs high-quality physically based materials from multi-view images.<n>Our results outperform state-of-the-art methods in material disentanglement on both synthetic and real scenes.
  arXiv  Detail & Related papers  (2025-12-15T10:05:59Z)
• MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference [83.38607296779423]
  We show that multi-view consistent material inference with more physically-based environment modeling is key to learning accurate reflections with Gaussian Splatting.<n>Our method faithfully recovers both illumination and geometry, achieving state-of-the-art rendering quality in novel views synthesis.
  arXiv  Detail & Related papers  (2025-10-13T13:29:20Z)
• HumanMaterial: Human Material Estimation from a Single Image via Progressive Training [20.601524328351665]
  Full-body Human inverse rendering aims to acquire high-quality materials, which helps achieve photo-realistic rendering under arbitrary illuminations.<n>Previous works alleviated this problem by building material dataset for training, but their simplified material data and rendering equation lead to rendering results with limited realism, especially that of skin.<n>We construct a higher-quality dataset (OpenHumanBRDF) based on scanned real data and statistical material data.<n>In addition to the normal, diffuse albedo, roughness, specular albedo, we produce displacement and subsurface scattering to enhance the realism of rendering results, especially for the skin.
  arXiv  Detail & Related papers  (2025-07-24T12:59:42Z)
• MatDecompSDF: High-Fidelity 3D Shape and PBR Material Decomposition from Multi-View Images [20.219010684946888]
  MatDecompSDF is a framework for recovering high-fidelity 3D shapes and decomposing their physically-based material properties from multi-view images.<n>Our method produces editable and relightable assets that can be seamlessly integrated into standard graphics pipelines.
  arXiv  Detail & Related papers  (2025-07-07T08:22:32Z)
• IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminations [64.07859467542664]
  Capturing geometric and material information from images remains a fundamental challenge in computer vision and graphics.<n>Traditional optimization-based methods often require hours of computational time to reconstruct geometry, material properties, and environmental lighting from dense multi-view inputs.<n>We introduce IDArb, a diffusion-based model designed to perform intrinsic decomposition on an arbitrary number of images under varying illuminations.
  arXiv  Detail & Related papers  (2024-12-16T18:52:56Z)
• IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illumination [37.96484120807323]
  This paper aims to recover object materials from posed images captured under an unknown static lighting condition. We learn the material prior with a generative model for regularizing the optimization process. Experiments on real-world and synthetic datasets demonstrate that our approach achieves state-of-the-art performance on material recovery.
  arXiv  Detail & Related papers  (2024-04-17T17:45:08Z)
• Intrinsic Image Diffusion for Indoor Single-view Material Estimation [55.276815106443976]
  We present Intrinsic Image Diffusion, a generative model for appearance decomposition of indoor scenes. Given a single input view, we sample multiple possible material explanations represented as albedo, roughness, and metallic maps. Our method produces significantly sharper, more consistent, and more detailed materials, outperforming state-of-the-art methods by $1.5dB$ on PSNR and by $45%$ better FID score on albedo prediction.
  arXiv  Detail & Related papers  (2023-12-19T15:56:19Z)
• DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer [78.91753256634453]
  We consider the challenging problem of predicting intrinsic object properties from a single image by exploiting differentiables. In this work, we propose DIBR++, a hybrid differentiable which supports these effects by combining specularization and ray-tracing. Compared to more advanced physics-based differentiables, DIBR++ is highly performant due to its compact and expressive model.
  arXiv  Detail & Related papers  (2021-10-30T01:59:39Z)
• MaterialGAN: Reflectance Capture using a Generative SVBRDF Model [33.578080406338266]
  We present MaterialGAN, a deep generative convolutional network based on StyleGAN2. We show that MaterialGAN can be used as a powerful material prior in an inverse rendering framework. We demonstrate this framework on the task of reconstructing SVBRDFs from images captured under flash illumination using a hand-held mobile phone.
  arXiv  Detail & Related papers  (2020-09-30T21:33:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.