MatPedia: A Universal Generative Foundation for High-Fidelity Material Synthesis

• URL: http://arxiv.org/abs/2511.16957v1
• Date: Fri, 21 Nov 2025 05:16:26 GMT
• Title: MatPedia: A Universal Generative Foundation for High-Fidelity Material Synthesis
• Authors: Di Luo, Shuhui Yang, Mingxin Yang, Jiawei Lu, Yixuan Tang, Xintong Han, Zhuo Chen, Beibei Wang, Chunchao Guo,
• Abstract summary: MatPedia is a foundation model built upon a novel joint RGB-PBR representation.<n>It compactly encodes materials into two latents: one for RGB appearance and one for the four PBR maps.<n>Trained on MatHybrid-410K, a mixed corpus combining PBR datasets with large-scale RGB images, MatPedia achieves native $1024times1024$ synthesis.
• Score: 29.919740823136163
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Physically-based rendering (PBR) materials are fundamental to photorealistic graphics, yet their creation remains labor-intensive and requires specialized expertise. While generative models have advanced material synthesis, existing methods lack a unified representation bridging natural image appearance and PBR properties, leading to fragmented task-specific pipelines and inability to leverage large-scale RGB image data. We present MatPedia, a foundation model built upon a novel joint RGB-PBR representation that compactly encodes materials into two interdependent latents: one for RGB appearance and one for the four PBR maps encoding complementary physical properties. By formulating them as a 5-frame sequence and employing video diffusion architectures, MatPedia naturally captures their correlations while transferring visual priors from RGB generation models. This joint representation enables a unified framework handling multiple material tasks--text-to-material generation, image-to-material generation, and intrinsic decomposition--within a single architecture. Trained on MatHybrid-410K, a mixed corpus combining PBR datasets with large-scale RGB images, MatPedia achieves native $1024\times1024$ synthesis that substantially surpasses existing approaches in both quality and diversity.

Related papers

• LumiTex: Towards High-Fidelity PBR Texture Generation with Illumination Context [40.98472333599951]
  Physically-based rendering (PBR) provides a principled standard for realistic material-lighting interactions in computer graphics.<n>LumiTex is an end-to-end framework that disentangles albedo and metallic-roughness under shared illumination priors.<n>LumiTex achieves state-of-the-art performance in texture quality, surpassing both existing open-source and commercial methods.
  arXiv  Detail & Related papers  (2025-11-24T18:59:58Z)
• DiffTex: Differentiable Texturing for Architectural Proxy Models [63.370581207280004]
  We propose an automated method for generating realistic texture maps for architectural proxy models at the texel level from unordered photographs.<n>Our approach establishes correspondences between texels on a UV map and pixels in the input images, with each texel's color computed as a weighted blend of associated pixel values.
  arXiv  Detail & Related papers  (2025-09-27T14:39:53Z)
• Large Material Gaussian Model for Relightable 3D Generation [54.10879517395551]
  We introduce a novel framework designed to generate high-quality 3D content with Physically Based Rendering (PBR) materials.<n>Our method not only exhibit greater visual appeal compared to baseline methods but also enhance material modeling, thereby enabling practical downstream rendering applications.
  arXiv  Detail & Related papers  (2025-09-26T09:35:12Z)
• PBR-SR: Mesh PBR Texture Super Resolution from 2D Image Priors [52.28858915766172]
  PBR-SR is a novel method for physically based rendering (PBR) texture super resolution (SR)<n>It outputs high-resolution, high-quality PBR textures from low-resolution (LR) PBR input in a zero-shot manner.
  arXiv  Detail & Related papers  (2025-06-03T13:15:34Z)
• ePBR: Extended PBR Materials in Image Synthesis [2.3241174970798126]
  Intrinsic image representation offers a well-balanced trade-off, decomposing images into fundamental components.<n>We extend intrinsic image representations to incorporate both reflection and transmission properties.<n>With the Extended PBR (ePBR) Materials, we can effectively edit the materials with precise controls.
  arXiv  Detail & Related papers  (2025-04-23T19:15:42Z)
• IntrinsiX: High-Quality PBR Generation using Image Priors [49.90007540430264]
  We introduce IntrinsiX, a novel method that generates high-quality intrinsic images from text description.<n>In contrast to existing text-to-image models whose outputs contain baked-in scene lighting, our approach predicts physically-based rendering (PBR) maps.
  arXiv  Detail & Related papers  (2025-04-01T17:47:48Z)
• MaterialMVP: Illumination-Invariant Material Generation via Multi-view PBR Diffusion [37.596740171045845]
  Physically-based rendering (PBR) has become a cornerstone in modern computer graphics, enabling realistic material representation and lighting interactions in 3D scenes.<n>We present a novel end-to-end model for generating PBR textures from 3D meshes and image prompts, addressing key challenges in multi-view material synthesis.
  arXiv  Detail & Related papers  (2025-03-13T11:57:30Z)
• MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models [42.42328559042189]
  MatCLIP is a novel method that extracts shape- and lighting-insensitive descriptors of PBR materials to assign plausible textures to 3D objects based on images.<n>By extending an Alpha-CLIP-based model on material renderings across diverse shapes and lighting, our approach generates descriptors that bridge the domains of PBR representations with photographs or renderings.<n>MatCLIP achieves a top-1 classification accuracy of 76.6%, outperforming state-of-the-art methods such as PhotoShape and MatAtlas.
  arXiv  Detail & Related papers  (2025-01-27T12:08:52Z)
• Image-GS: Content-Adaptive Image Representation via 2D Gaussians [52.598772767324036]
  We introduce Image-GS, a content-adaptive image representation based on 2D Gaussians radiance.<n>It supports hardware-friendly rapid access for real-time usage, requiring only 0.3K MACs to decode a pixel.<n>We demonstrate its versatility with several applications, including texture compression, semantics-aware compression, and joint image compression and restoration.
  arXiv  Detail & Related papers  (2024-07-02T00:45:21Z)
• MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generation [54.64194935409982]
  We introduce MuLAn: a novel dataset comprising over 44K MUlti-Layer-wise RGBA decompositions. MuLAn is the first photorealistic resource providing instance decomposition and spatial information for high quality images. We aim to encourage the development of novel generation and editing technology, in particular layer-wise solutions.
  arXiv  Detail & Related papers  (2024-04-03T14:58:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.