3D Gaussian Inpainting with Depth-Guided Cross-View Consistency

• URL: http://arxiv.org/abs/2502.11801v2
• Date: Mon, 28 Apr 2025 09:14:48 GMT
• Title: 3D Gaussian Inpainting with Depth-Guided Cross-View Consistency
• Authors: Sheng-Yu Huang, Zi-Ting Chou, Yu-Chiang Frank Wang,
• Abstract summary: We propose a framework of 3D Gaussian Inpainting with Depth-Guided Cross-View Consistency (3DGIC) for cross-view consistent 3D inpainting.<n>Our 3DGIC exploits background pixels visible across different views for updating the inpainting mask, allowing us to refine the 3DGS for inpainting purposes.
• Score: 30.951440204237166
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When performing 3D inpainting using novel-view rendering methods like Neural Radiance Field (NeRF) or 3D Gaussian Splatting (3DGS), how to achieve texture and geometry consistency across camera views has been a challenge. In this paper, we propose a framework of 3D Gaussian Inpainting with Depth-Guided Cross-View Consistency (3DGIC) for cross-view consistent 3D inpainting. Guided by the rendered depth information from each training view, our 3DGIC exploits background pixels visible across different views for updating the inpainting mask, allowing us to refine the 3DGS for inpainting purposes.Through extensive experiments on benchmark datasets, we confirm that our 3DGIC outperforms current state-of-the-art 3D inpainting methods quantitatively and qualitatively.

Related papers

• WarpGAN: Warping-Guided 3D GAN Inversion with Style-Based Novel View Inpainting [68.77882703764142]
  3D GAN inversion projects a single image into the latent space of a pre-trained 3D GAN to achieve single-shot novel view synthesis.<n>We introduce the warping-and-inpainting strategy to incorporate image inpainting into 3D GAN inversion and propose a novel 3D GAN inversion method, WarpGAN.
  arXiv  Detail & Related papers  (2025-11-11T12:42:07Z)
• Perspective-aware 3D Gaussian Inpainting with Multi-view Consistency [22.654962078504017]
  We present PAInpainter, a novel approach designed to advance 3D Gaussian inpainting by leveraging perspective-aware content propagation and consistency verification across multi-view inpainted images.<n>Our approach achieves superior 3D inpainting quality, with PSNR scores of 26.03 dB and 29.51 dB on the SPIn-NeRF and NeRFiller datasets, respectively.
  arXiv  Detail & Related papers  (2025-10-13T04:10:39Z)
• SplatFill: 3D Scene Inpainting via Depth-Guided Gaussian Splatting [17.45252036814217]
  3D Gaussian Splatting (3DGS) has enabled the creation of highly realistic 3D scene representations from sets of multi-view images.<n>In this work, we introduce SplatFill, a novel depth-guided approach for 3DGS scene inpainting that state-of-the-art perceptual quality and improved efficiency.
  arXiv  Detail & Related papers  (2025-09-09T14:47:47Z)
• DiGA3D: Coarse-to-Fine Diffusional Propagation of Geometry and Appearance for Versatile 3D Inpainting [10.515239541326737]
  Single reference inpainting methods lack robustness when dealing with views far from the reference view.<n>Appearance inconsistency arises when independently inpainting multi-view images with 2D diffusion priors.<n>DiGA3D uses diffusion models to propagate consistent appearance and geometry in a coarse-to-fine manner.
  arXiv  Detail & Related papers  (2025-07-01T04:57:08Z)
• Visibility-Uncertainty-guided 3D Gaussian Inpainting via Scene Conceptional Learning [63.94919846010485]
  3D Gaussian inpainting (3DGI) is challenging in effectively leveraging complementary visual and semantic cues from multiple input views. We propose a method that measures the visibility uncertainties of 3D points across different input views and uses them to guide 3DGI. We build a novel 3DGI framework, VISTA, by integrating VISibility-uncerTainty-guided 3DGI with scene conceptuAl learning.
  arXiv  Detail & Related papers  (2025-04-23T06:21:11Z)
• EVolSplat: Efficient Volume-based Gaussian Splatting for Urban View Synthesis [61.1662426227688]
  Existing NeRF and 3DGS-based methods show promising results in achieving photorealistic renderings but require slow, per-scene optimization. We introduce EVolSplat, an efficient 3D Gaussian Splatting model for urban scenes that works in a feed-forward manner.
  arXiv  Detail & Related papers  (2025-03-26T02:47:27Z)
• MVS-GS: High-Quality 3D Gaussian Splatting Mapping via Online Multi-View Stereo [9.740087094317735]
  We propose a novel framework for high-quality 3DGS modeling using an online multi-view stereo approach.<n>Our method estimates MVS depth using sequential frames from a local time window and applies comprehensive depth refinement techniques.<n> Experimental results demonstrate that our method outperforms state-of-the-art dense SLAM methods.
  arXiv  Detail & Related papers  (2024-12-26T09:20:04Z)
• 3D-HGS: 3D Half-Gaussian Splatting [5.766096863155448]
  Photo-realistic image rendering from scene 3D reconstruction is a fundamental problem in 3D computer vision. We introduce 3D Half-Gaussian kernels, which can be used as a plug-and-play kernel.
  arXiv  Detail & Related papers  (2024-06-04T19:04:29Z)
• ART3D: 3D Gaussian Splatting for Text-Guided Artistic Scenes Generation [18.699440994076003]
  ART3D is a novel framework that combines diffusion models and 3D Gaussian splatting techniques. By leveraging depth information and an initial artistic image, we generate a point cloud map. We also propose a depth consistency module to enhance 3D scene consistency.
  arXiv  Detail & Related papers  (2024-05-17T03:19:36Z)
• InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Prior [36.23604779569843]
  3D Gaussians have recently emerged as an efficient representation for novel view synthesis. This work studies its editability with a particular focus on the inpainting task. Compared to 2D inpainting, the crux of inpainting 3D Gaussians is to figure out the rendering-relevant properties of the introduced points.
  arXiv  Detail & Related papers  (2024-04-17T17:59:53Z)
• Sketch3D: Style-Consistent Guidance for Sketch-to-3D Generation [55.73399465968594]
  This paper proposes a novel generation paradigm Sketch3D to generate realistic 3D assets with shape aligned with the input sketch and color matching the textual description. Three strategies are designed to optimize 3D Gaussians, i.e., structural optimization via a distribution transfer mechanism, color optimization with a straightforward MSE loss and sketch similarity optimization with a CLIP-based geometric similarity loss.
  arXiv  Detail & Related papers  (2024-04-02T11:03:24Z)
• Neural 3D Strokes: Creating Stylized 3D Scenes with Vectorized 3D Strokes [20.340259111585873]
  We present Neural 3D Strokes, a novel technique to generate stylized images of a 3D scene at arbitrary novel views from multi-view 2D images. Our approach draws inspiration from image-to-painting methods, simulating the progressive painting process of human artwork with vector strokes.
  arXiv  Detail & Related papers  (2023-11-27T09:02:21Z)
• High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization [51.878078860524795]
  We present a high-fidelity 3D generative adversarial network (GAN) inversion framework that can synthesize photo-realistic novel views. Our approach enables high-fidelity 3D rendering from a single image, which is promising for various applications of AI-generated 3D content.
  arXiv  Detail & Related papers  (2022-11-28T18:59:52Z)
• Geometric Correspondence Fields: Learned Differentiable Rendering for 3D Pose Refinement in the Wild [96.09941542587865]
  We present a novel 3D pose refinement approach based on differentiable rendering for objects of arbitrary categories in the wild. In this way, we precisely align 3D models to objects in RGB images which results in significantly improved 3D pose estimates. We evaluate our approach on the challenging Pix3D dataset and achieve up to 55% relative improvement compared to state-of-the-art refinement methods in multiple metrics.
  arXiv  Detail & Related papers  (2020-07-17T12:34:38Z)
• 3D Photography using Context-aware Layered Depth Inpainting [50.66235795163143]
  We propose a method for converting a single RGB-D input image into a 3D photo. A learning-based inpainting model synthesizes new local color-and-depth content into the occluded region. The resulting 3D photos can be efficiently rendered with motion parallax.
  arXiv  Detail & Related papers  (2020-04-09T17:59:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.