MVPainter: Accurate and Detailed 3D Texture Generation via Multi-View Diffusion with Geometric Control

• URL: http://arxiv.org/abs/2505.12635v1
• Date: Mon, 19 May 2025 02:40:24 GMT
• Title: MVPainter: Accurate and Detailed 3D Texture Generation via Multi-View Diffusion with Geometric Control
• Authors: Mingqi Shao, Feng Xiong, Zhaoxu Sun, Mu Xu,
• Abstract summary: We investigate 3D texture generation through the lens of three core dimensions: reference-texture alignment, geometry-texture consistency, and local texture quality.<n>We propose MVPainter, which employs data filtering and augmentation strategies to enhance texture fidelity and detail.<n>We extract physically-based rendering (PBR) attributes from the generated views to produce PBR meshes suitable for real-world rendering applications.
• Score: 1.8463601973573158
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, significant advances have been made in 3D object generation. Building upon the generated geometry, current pipelines typically employ image diffusion models to generate multi-view RGB images, followed by UV texture reconstruction through texture baking. While 3D geometry generation has improved significantly, supported by multiple open-source frameworks, 3D texture generation remains underexplored. In this work, we systematically investigate 3D texture generation through the lens of three core dimensions: reference-texture alignment, geometry-texture consistency, and local texture quality. To tackle these issues, we propose MVPainter, which employs data filtering and augmentation strategies to enhance texture fidelity and detail, and introduces ControlNet-based geometric conditioning to improve texture-geometry alignment. Furthermore, we extract physically-based rendering (PBR) attributes from the generated views to produce PBR meshes suitable for real-world rendering applications. MVPainter achieves state-of-the-art results across all three dimensions, as demonstrated by human-aligned evaluations. To facilitate further research and reproducibility, we also release our full pipeline as an open-source system, including data construction, model architecture, and evaluation tools.

Related papers

• End-to-End Fine-Tuning of 3D Texture Generation using Differentiable Rewards [8.953379216683732]
  We propose an end-to-end differentiable, reinforcement-learning-free framework that embeds human feedback, expressed as differentiable reward functions, directly into the 3D texture pipeline.<n>By back-propagating preference signals through both geometric and appearance modules, our method generates textures that respect the 3D geometry structure and align with desired criteria.
  arXiv  Detail & Related papers  (2025-06-23T06:24:12Z)
• UniTEX: Universal High Fidelity Generative Texturing for 3D Shapes [35.667175445637604]
  We present UniTEX, a novel two-stage 3D texture generation framework.<n>UniTEX achieves superior visual quality and texture integrity compared to existing approaches.
  arXiv  Detail & Related papers  (2025-05-29T08:58:41Z)
• PacTure: Efficient PBR Texture Generation on Packed Views with Visual Autoregressive Models [73.4445896872942]
  PacTure is a framework for generating physically-based rendering (PBR) material textures from an un-domain 3D mesh.<n>We introduce view packing, a novel technique that increases the effective resolution for each view.
  arXiv  Detail & Related papers  (2025-05-28T14:23:30Z)
• RomanTex: Decoupling 3D-aware Rotary Positional Embedded Multi-Attention Network for Texture Synthesis [10.350576861948952]
  RomanTex is a multiview-based texture generation framework that integrates a multi-attention network with an underlying 3D representation.<n>Our method achieves state-of-the-art results in texture quality and consistency.
  arXiv  Detail & Related papers  (2025-03-24T17:56:11Z)
• Pandora3D: A Comprehensive Framework for High-Quality 3D Shape and Texture Generation [56.862552362223425]
  This report presents a comprehensive framework for generating high-quality 3D shapes and textures from diverse input prompts.<n>The framework consists of 3D shape generation and texture generation.<n>This report details the system architecture, experimental results, and potential future directions to improve and expand the framework.
  arXiv  Detail & Related papers  (2025-02-20T04:22:30Z)
• GraphicsDreamer: Image to 3D Generation with Physical Consistency [32.26851174969898]
  We introduce GraphicsDreamer, a method for creating highly usable 3D meshes from single images.<n>In the geometry fusion stage, we continue to enforce the PBR constraints, ensuring that the generated 3D objects possess reliable texture details.<n>Our method incorporates topology optimization and fast UV unwrapping capabilities, allowing the 3D products to be seamlessly imported into graphics engines.
  arXiv  Detail & Related papers  (2024-12-18T10:01:27Z)
• Textured Mesh Saliency: Bridging Geometry and Texture for Human Perception in 3D Graphics [50.23625950905638]
  We present a new dataset for textured mesh saliency, created through an innovative eye-tracking experiment in a six degrees of freedom (6-DOF) VR environment.<n>Our proposed model predicts saliency maps for textured mesh surfaces by treating each triangular face as an individual unit and assigning a saliency density value to reflect the importance of each local surface region.
  arXiv  Detail & Related papers  (2024-12-11T08:27:33Z)
• Direct and Explicit 3D Generation from a Single Image [25.207277983430608]
  We introduce a novel framework to directly generate explicit surface geometry and texture using multi-view 2D depth and RGB images. We incorporate epipolar attention into the latent-to-pixel decoder for pixel-level multi-view consistency. By back-projecting the generated depth pixels into 3D space, we create a structured 3D representation.
  arXiv  Detail & Related papers  (2024-11-17T03:14:50Z)
• DreamPolish: Domain Score Distillation With Progressive Geometry Generation [66.94803919328815]
  We introduce DreamPolish, a text-to-3D generation model that excels in producing refined geometry and high-quality textures. In the geometry construction phase, our approach leverages multiple neural representations to enhance the stability of the synthesis process. In the texture generation phase, we introduce a novel score distillation objective, namely domain score distillation (DSD), to guide neural representations toward such a domain.
  arXiv  Detail & Related papers  (2024-11-03T15:15:01Z)
• Meta 3D TextureGen: Fast and Consistent Texture Generation for 3D Objects [54.80813150893719]
  We introduce Meta 3D TextureGen: a new feedforward method comprised of two sequential networks aimed at generating high-quality textures in less than 20 seconds. Our method state-of-the-art results in quality and speed by conditioning a text-to-image model on 3D semantics in 2D space and fusing them into a complete and high-resolution UV texture map. In addition, we introduce a texture enhancement network that is capable of up-scaling any texture by an arbitrary ratio, producing 4k pixel resolution textures.
  arXiv  Detail & Related papers  (2024-07-02T17:04:34Z)
• High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization [51.878078860524795]
  We present a high-fidelity 3D generative adversarial network (GAN) inversion framework that can synthesize photo-realistic novel views. Our approach enables high-fidelity 3D rendering from a single image, which is promising for various applications of AI-generated 3D content.
  arXiv  Detail & Related papers  (2022-11-28T18:59:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.