3D PixBrush: Image-Guided Local Texture Synthesis

• URL: http://arxiv.org/abs/2507.03731v1
• Date: Fri, 04 Jul 2025 17:38:34 GMT
• Title: 3D PixBrush: Image-Guided Local Texture Synthesis
• Authors: Dale Decatur, Itai Lang, Kfir Aberman, Rana Hanocka,
• Abstract summary: 3D PixBrush predicts a localization mask and a synthesized texture that faithfully portray the object in the reference image.<n>Our method produces masks that conform to the geometry of the reference image.
• Score: 19.866883340848982
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present 3D PixBrush, a method for performing image-driven edits of local regions on 3D meshes. 3D PixBrush predicts a localization mask and a synthesized texture that faithfully portray the object in the reference image. Our predicted localizations are both globally coherent and locally precise. Globally - our method contextualizes the object in the reference image and automatically positions it onto the input mesh. Locally - our method produces masks that conform to the geometry of the reference image. Notably, our method does not require any user input (in the form of scribbles or bounding boxes) to achieve accurate localizations. Instead, our method predicts a localization mask on the 3D mesh from scratch. To achieve this, we propose a modification to the score distillation sampling technique which incorporates both the predicted localization and the reference image, referred to as localization-modulated image guidance. We demonstrate the effectiveness of our proposed technique on a wide variety of meshes and images.

Related papers

• Self-supervised Learning of Neural Implicit Feature Fields for Camera Pose Refinement [32.335953514942474]
  This paper proposes to jointly learn the scene representation along with a 3D dense feature field and a 2D feature extractor. We learn the underlying geometry of the scene with an implicit field through volumetric rendering and design our feature field to leverage intermediate geometric information encoded in the implicit field. Visual localization is then achieved by aligning the image-based features and the rendered volumetric features.
  arXiv  Detail & Related papers  (2024-06-12T17:51:53Z)
• Dream-in-Style: Text-to-3D Generation Using Stylized Score Distillation [14.079043195485601]
  We present a method to generate 3D objects in styles.<n>Our method takes a text prompt and a style reference image as input and reconstructs a neural radiance field to synthesize a 3D model.
  arXiv  Detail & Related papers  (2024-06-05T16:27:34Z)
• ShapeFusion: A 3D diffusion model for localized shape editing [37.82690898932135]
  We propose an effective diffusion masking training strategy that, by design, facilitates localized manipulation of any shape region. Compared to the current state-of-the-art our method leads to more interpretable shape manipulations than methods relying on latent code state.
  arXiv  Detail & Related papers  (2024-03-28T18:50:19Z)
• SceneWiz3D: Towards Text-guided 3D Scene Composition [134.71933134180782]
  Existing approaches either leverage large text-to-image models to optimize a 3D representation or train 3D generators on object-centric datasets. We introduce SceneWiz3D, a novel approach to synthesize high-fidelity 3D scenes from text.
  arXiv  Detail & Related papers  (2023-12-13T18:59:30Z)
• ConTex-Human: Free-View Rendering of Human from a Single Image with Texture-Consistent Synthesis [49.28239918969784]
  We introduce a texture-consistent back view synthesis module that could transfer the reference image content to the back view. We also propose a visibility-aware patch consistency regularization for texture mapping and refinement combined with the synthesized back view texture.
  arXiv  Detail & Related papers  (2023-11-28T13:55:53Z)
• 3D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillation [21.703142822709466]
  3D Paintbrush is a technique for automatically local semantic regions on meshes via text descriptions. Our method is designed to operate directly on meshes, producing texture maps seamlessly.
  arXiv  Detail & Related papers  (2023-11-16T05:13:44Z)
• 3DStyle-Diffusion: Pursuing Fine-grained Text-driven 3D Stylization with 2D Diffusion Models [102.75875255071246]
  3D content creation via text-driven stylization has played a fundamental challenge to multimedia and graphics community. We propose a new 3DStyle-Diffusion model that triggers fine-grained stylization of 3D meshes with additional controllable appearance and geometric guidance from 2D Diffusion models.
  arXiv  Detail & Related papers  (2023-11-09T15:51:27Z)
• Generating Texture for 3D Human Avatar from a Single Image using Sampling and Refinement Networks [8.659903550327442]
  We propose a texture synthesis method for a 3D human avatar that incorporates geometry information. A sampler network fills in the occluded regions of the source image and aligns the texture with the surface of the target 3D mesh. To maintain the clear details in the given image, both sampled and refined texture is blended to produce the final texture map.
  arXiv  Detail & Related papers  (2023-05-01T16:44:02Z)
• TMO: Textured Mesh Acquisition of Objects with a Mobile Device by using Differentiable Rendering [54.35405028643051]
  We present a new pipeline for acquiring a textured mesh in the wild with a single smartphone. Our method first introduces an RGBD-aided structure from motion, which can yield filtered depth maps. We adopt the neural implicit surface reconstruction method, which allows for high-quality mesh.
  arXiv  Detail & Related papers  (2023-03-27T10:07:52Z)
• Vision Transformer for NeRF-Based View Synthesis from a Single Input Image [49.956005709863355]
  We propose to leverage both the global and local features to form an expressive 3D representation. To synthesize a novel view, we train a multilayer perceptron (MLP) network conditioned on the learned 3D representation to perform volume rendering. Our method can render novel views from only a single input image and generalize across multiple object categories using a single model.
  arXiv  Detail & Related papers  (2022-07-12T17:52:04Z)
• Realistic Image Synthesis with Configurable 3D Scene Layouts [59.872657806747576]
  We propose a novel approach to realistic-looking image synthesis based on a 3D scene layout. Our approach takes a 3D scene with semantic class labels as input and trains a 3D scene painting network. With the trained painting network, realistic-looking images for the input 3D scene can be rendered and manipulated.
  arXiv  Detail & Related papers  (2021-08-23T09:44:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.