OmniFabric: Coherent UV Space Texture Synthesis for 3D Garment Reconstruction
Abstract Overview
OmniFabric is a method for generating simulation-ready garment textures from a single in-the-wild clothing image by synthesizing textures directly in 2D sewing-pattern UV space. The pipeline first predicts sewing patterns and a canonical 3D garment, reposes the input garment to align with that geometry, and uses generated multi-view imagery to build a globally aligned but coarse UV texture map. It then refines that map with a diffusion-transformer-based normalization model conditioned on the coarse texture, a frontal render, and a 3D position map. The goal is to remove baked-in lighting, wrinkles, projection distortions, and missing regions while preserving the garment’s global design and fine details.
Novelty
The paper’s main distinction is shifting garment texture synthesis from local patch extraction or direct 3D painting to globally coherent synthesis in canonical sewing-pattern UV space. It combines VLM-guided pose alignment, video-model-based multi-view completion, and a UV-space diffusion transformer trained on automatically generated synthetic sewing-pattern data.
Results
The paper reports that OmniFabric outperforms FabricDiffusion, Paint3D, and Hunyuan3D-2.0 on the synthetic benchmark across LPIPS, SSIM, MS-SSIM, DISTS, and CLIP-score, with the reported best values including LPIPS 0.092, SSIM 0.868, MS-SSIM 0.905, DISTS 0.121, and CLIP-s 0.963. A user study on real-world images also ranks OmniFabric best for overall quality, fidelity, back-view plausibility, and global coherence. Additional analyses indicate that the UV-space normalization stage reduces accumulated errors and that an open-source variant achieves similar performance to the default setup.
Key Points
- OmniFabric synthesizes garment textures in sewing-pattern UV space rather than directly painting draped 3D geometry, aiming to preserve global layout while avoiding distortion and baked-in illumination artifacts.
- The method uses a two-stage pipeline: coarse texture initialization from pose-aligned and multi-view generated imagery, followed by DiT-based UV refinement conditioned on coarse texture, frontal render, and 3D positional features.
- Experiments show stronger quantitative performance than prior baselines on a synthetic dataset and better human preference on real images, while ablations support the importance of the coarse map, frontal view condition, and position map.