MatLat: Material Latent Space for PBR Texture Generation
- URL: http://arxiv.org/abs/2512.17302v1
- Date: Fri, 19 Dec 2025 07:35:09 GMT
- Title: MatLat: Material Latent Space for PBR Texture Generation
- Authors: Kyeongmin Yeo, Yunhong Min, Jaihoon Kim, Minhyuk Sung,
- Abstract summary: We propose a generative framework for producing high-quality PBR textures on a given 3D mesh.<n>As large-scale PBR texture datasets are scarce, our approach focuses on effectively leveraging the embedding space and diffusion priors of pretrained latent image generative models.
- Score: 27.611659308292506
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We propose a generative framework for producing high-quality PBR textures on a given 3D mesh. As large-scale PBR texture datasets are scarce, our approach focuses on effectively leveraging the embedding space and diffusion priors of pretrained latent image generative models while learning a material latent space, MatLat, through targeted fine-tuning. Unlike prior methods that freeze the embedding network and thus lead to distribution shifts when encoding additional PBR channels and hinder subsequent diffusion training, we fine-tune the pretrained VAE so that new material channels can be incorporated with minimal latent distribution deviation. We further show that correspondence-aware attention alone is insufficient for cross-view consistency unless the latent-to-image mapping preserves locality. To enforce this locality, we introduce a regularization in the VAE fine-tuning that crops latent patches, decodes them, and aligns the corresponding image regions to maintain strong pixel-latent spatial correspondence. Ablation studies and comparison with previous baselines demonstrate that our framework improves PBR texture fidelity and that each component is critical for achieving state-of-the-art performance.
Related papers
- Chord: Chain of Rendering Decomposition for PBR Material Estimation from Generated Texture Images [10.46170854352924]
We propose a novel two-stage generate-and-estimate framework for PBR material generation.<n>In the generation stage, a fine-tuned diffusion model synthesizes shaded, tileable texture images aligned with user input.<n>In the estimation stage, we introduce a chained decomposition scheme that sequentially predicts SVBRDF channels by passing previously extracted representation as input into a single-step image-conditional diffusion model.
arXiv Detail & Related papers (2025-09-12T04:03:07Z) - Learning Deblurring Texture Prior from Unpaired Data with Diffusion Model [92.61216319417208]
We propose a novel diffusion model (DM)-based framework, dubbed ours, for image deblurring.<n>ours performs DM to generate the prior knowledge that aids in recovering the textures of blurry images.<n>To fully exploit the generated texture priors, we present the Texture Transfer Transformer layer (TTformer)
arXiv Detail & Related papers (2025-07-18T01:50:31Z) - PBR-SR: Mesh PBR Texture Super Resolution from 2D Image Priors [52.28858915766172]
PBR-SR is a novel method for physically based rendering (PBR) texture super resolution (SR)<n>It outputs high-resolution, high-quality PBR textures from low-resolution (LR) PBR input in a zero-shot manner.
arXiv Detail & Related papers (2025-06-03T13:15:34Z) - FlexPainter: Flexible and Multi-View Consistent Texture Generation [15.727635740684157]
textbfFlexPainter is a novel texture generation pipeline that enables flexible multi-modal conditional guidance.<n>Our framework significantly outperforms state-of-the-art methods in both flexibility and generation quality.
arXiv Detail & Related papers (2025-06-03T08:36:03Z) - PacTure: Efficient PBR Texture Generation on Packed Views with Visual Autoregressive Models [73.4445896872942]
PacTure is a framework for generating physically-based rendering (PBR) material textures from an un-domain 3D mesh.<n>We introduce view packing, a novel technique that increases the effective resolution for each view.
arXiv Detail & Related papers (2025-05-28T14:23:30Z) - IntrinsiX: High-Quality PBR Generation using Image Priors [49.90007540430264]
We introduce IntrinsiX, a novel method that generates high-quality intrinsic images from text description.<n>In contrast to existing text-to-image models whose outputs contain baked-in scene lighting, our approach predicts physically-based rendering (PBR) maps.
arXiv Detail & Related papers (2025-04-01T17:47:48Z) - Coherent and Multi-modality Image Inpainting via Latent Space Optimization [61.99406669027195]
PILOT (intextbfPainting vtextbfIa textbfLatent textbfOptextbfTimization) is an optimization approach grounded on a novel textitsemantic centralization and textitbackground preservation loss.
Our method searches latent spaces capable of generating inpainted regions that exhibit high fidelity to user-provided prompts while maintaining coherence with the background.
arXiv Detail & Related papers (2024-07-10T19:58:04Z) - FreePIH: Training-Free Painterly Image Harmonization with Diffusion
Model [19.170302996189335]
Our FreePIH method tames the denoising process as a plug-in module for foreground image style transfer.
We make use of multi-scale features to enforce the consistency of the content and stability of the foreground objects in the latent space.
Our method can surpass representative baselines by large margins.
arXiv Detail & Related papers (2023-11-25T04:23:49Z) - Image Fine-grained Inpainting [89.17316318927621]
We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields.
To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss.
We also employ a discriminator with local and global branches to ensure local-global contents consistency.
arXiv Detail & Related papers (2020-02-07T03:45:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.