Distractor-free Generalizable 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2411.17605v2
• Date: Mon, 02 Jun 2025 05:22:27 GMT
• Title: Distractor-free Generalizable 3D Gaussian Splatting
• Authors: Yanqi Bao, Jing Liao, Jing Huo, Yang Gao,
• Abstract summary: We present DGGS, a novel framework that addresses the previously unexplored challenge: $textbfDistractor-free Generalizable 3D Gaussian Splatting$ (3DGS)<n>It mitigates 3D inconsistency and training instability caused by distractor data in the cross-scenes generalizable train setting.<n>Our generalizable mask prediction even achieves an accuracy superior to existing scene-specific training methods.
• Score: 26.762275313390194
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present DGGS, a novel framework that addresses the previously unexplored challenge: $\textbf{Distractor-free Generalizable 3D Gaussian Splatting}$ (3DGS). It mitigates 3D inconsistency and training instability caused by distractor data in the cross-scenes generalizable train setting while enabling feedforward inference for 3DGS and distractor masks from references in the unseen scenes. To achieve these objectives, DGGS proposes a scene-agnostic reference-based mask prediction and refinement module during the training phase, effectively eliminating the impact of distractor on training stability. Moreover, we combat distractor-induced artifacts and holes at inference time through a novel two-stage inference framework for references scoring and re-selection, complemented by a distractor pruning mechanism that further removes residual distractor 3DGS-primitive influences. Extensive feedforward experiments on the real and our synthetic data show DGGS's reconstruction capability when dealing with novel distractor scenes. Moreover, our generalizable mask prediction even achieves an accuracy superior to existing scene-specific training methods. Homepage is https://github.com/bbbbby-99/DGGS.

Related papers

• RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS [79.15416002879239]
  3D Gaussian Splatting has gained significant attention for its real-time, photo-realistic rendering in novel-view synthesis and 3D modeling.<n>Existing methods struggle with accurately modeling scenes affected by transient objects, leading to artifacts in the rendered images.<n>We propose RobustSplat, a robust solution based on two critical designs.
  arXiv  Detail & Related papers  (2025-06-03T11:13:48Z)
• 3D Gaussian Splat Vulnerabilities [20.065766098524698]
  We introduce view-dependent Gaussian appearances to embed adversarial content visible only from specific viewpoints.<n>We demonstrate DAGGER, a targeted adversarial attack directly perturbing 3D Gaussians without access to underlying training data.<n>These attacks highlight underexplored vulnerabilities in 3DGS, introducing a new potential threat to robotic learning for autonomous navigation and other safety-critical 3DGS applications.
  arXiv  Detail & Related papers  (2025-05-30T22:21:22Z)
• Street Gaussians without 3D Object Tracker [86.62329193275916]
  Existing methods rely on labor-intensive manual labeling of object poses to reconstruct dynamic objects in canonical space.<n>We propose a stable object tracking module by leveraging associations from 2D deep trackers within a 3D object fusion strategy.<n>We address inevitable tracking errors by further introducing a motion learning strategy in an implicit feature space that autonomously corrects trajectory errors and recovers missed detections.
  arXiv  Detail & Related papers  (2024-12-07T05:49:42Z)
• A Lesson in Splats: Teacher-Guided Diffusion for 3D Gaussian Splats Generation with 2D Supervision [65.33043028101471]
  We introduce a diffusion model for Gaussian Splats, SplatDiffusion, to enable generation of three-dimensional structures from single images. Existing methods rely on deterministic, feed-forward predictions, which limit their ability to handle the inherent ambiguity of 3D inference from 2D data.
  arXiv  Detail & Related papers  (2024-12-01T00:29:57Z)
• T-3DGS: Removing Transient Objects for 3D Scene Reconstruction [83.05271859398779]
  Transient objects in video sequences can significantly degrade the quality of 3D scene reconstructions.<n>We propose T-3DGS, a novel framework that robustly filters out transient distractors during 3D reconstruction using Gaussian Splatting.
  arXiv  Detail & Related papers  (2024-11-29T07:45:24Z)
• Robust 3D Gaussian Splatting for Novel View Synthesis in Presence of Distractors [44.55317154371679]
  3D Gaussian Splatting has shown impressive novel view synthesis results. It is vulnerable to dynamic objects polluting the input data of an otherwise static scene, so called distractors. We show that our approach is robust to various distractors and strongly improves rendering quality on distractor-polluted scenes.
  arXiv  Detail & Related papers  (2024-08-21T15:21:27Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• SpotlessSplats: Ignoring Distractors in 3D Gaussian Splatting [44.42317312908314]
  3D Gaussian Splatting (3DGS) is a promising technique for 3D reconstruction, offering efficient training and rendering speeds. Current methods require highly controlled environments to meet the inter-view consistency assumption of 3DGS. We present SpotLessSplats, an approach that leverages pre-trained and general-purpose features coupled with robust optimization to effectively ignore transient distractors.
  arXiv  Detail & Related papers  (2024-06-28T17:07:11Z)
• Toward Availability Attacks in 3D Point Clouds [28.496421433836908]
  We show that extending 2D availability attacks directly to 3D point clouds under distance regularization is susceptible to the degeneracy. We propose a novel Feature Collision Error-Minimization (FC-EM) method, which creates additional shortcuts in the feature space. Experiments on typical point cloud datasets, 3D intracranial aneurysm medical dataset, and 3D face dataset verify the superiority and practicality of our approach.
  arXiv  Detail & Related papers  (2024-06-26T08:13:30Z)
• DPO: Dual-Perturbation Optimization for Test-time Adaptation in 3D Object Detection [34.04061546178302]
  Training 3D detectors in the real world often yields unsatisfactory performance when the distribution of the test data significantly deviates from the training data. We propose textbfdual-perturbation optimization (DPO) for textbfunderlineTest-underlinetime underlineAdaptation in underline3D. We minimize the sharpness to cultivate a flat loss landscape to ensure model resiliency to minor data variations.
  arXiv  Detail & Related papers  (2024-06-19T23:46:08Z)
• DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery [71.6345505427213]
  DPMesh is an innovative framework for occluded human mesh recovery. It capitalizes on the profound diffusion prior about object structure and spatial relationships embedded in a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-04-01T18:59:13Z)
• 2D Feature Distillation for Weakly- and Semi-Supervised 3D Semantic Segmentation [92.17700318483745]
  We propose an image-guidance network (IGNet) which builds upon the idea of distilling high level feature information from a domain adapted synthetically trained 2D semantic segmentation network. IGNet achieves state-of-the-art results for weakly-supervised LiDAR semantic segmentation on ScribbleKITTI, boasting up to 98% relative performance to fully supervised training with only 8% labeled points.
  arXiv  Detail & Related papers  (2023-11-27T07:57:29Z)
• ODM3D: Alleviating Foreground Sparsity for Semi-Supervised Monocular 3D Object Detection [15.204935788297226]
  ODM3D framework entails cross-modal knowledge distillation at various levels to inject LiDAR-domain knowledge into a monocular detector during training. By identifying foreground sparsity as the main culprit behind existing methods' suboptimal training, we exploit the precise localisation information embedded in LiDAR points. Our method ranks 1st in both KITTI validation and test benchmarks, significantly surpassing all existing monocular methods, supervised or semi-supervised.
  arXiv  Detail & Related papers  (2023-10-28T07:12:09Z)
• 3D Adversarial Augmentations for Robust Out-of-Domain Predictions [115.74319739738571]
  We focus on improving the generalization to out-of-domain data. We learn a set of vectors that deform the objects in an adversarial fashion. We perform adversarial augmentation by applying the learned sample-independent vectors to the available objects when training a model.
  arXiv  Detail & Related papers  (2023-08-29T17:58:55Z)
• DQS3D: Densely-matched Quantization-aware Semi-supervised 3D Detection [6.096961718434965]
  We study the problem of semi-supervised 3D object detection, which is of great importance considering the high annotation cost for cluttered 3D indoor scenes. We resort to the robust and principled framework of selfteaching, which has triggered notable progress for semisupervised learning recently. We propose the first semisupervised 3D detection algorithm that works in the singlestage manner and allows spatially dense training signals.
  arXiv  Detail & Related papers  (2023-04-25T17:59:54Z)
• Augment and Criticize: Exploring Informative Samples for Semi-Supervised Monocular 3D Object Detection [64.65563422852568]
  We improve the challenging monocular 3D object detection problem with a general semi-supervised framework. We introduce a novel, simple, yet effective Augment and Criticize' framework that explores abundant informative samples from unlabeled data. The two new detectors, dubbed 3DSeMo_DLE and 3DSeMo_FLEX, achieve state-of-the-art results with remarkable improvements for over 3.5% AP_3D/BEV (Easy) on KITTI.
  arXiv  Detail & Related papers  (2023-03-20T16:28:15Z)
• Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation [70.32536356351706]
  We introduce MRP-Net that constitutes a common deep network backbone with two output heads subscribing to two diverse configurations. We derive suitable measures to quantify prediction uncertainty at both pose and joint level. We present a comprehensive evaluation of the proposed approach and demonstrate state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-29T07:14:58Z)
• The KFIoU Loss for Rotated Object Detection [115.334070064346]
  In this paper, we argue that one effective alternative is to devise an approximate loss who can achieve trend-level alignment with SkewIoU loss. Specifically, we model the objects as Gaussian distribution and adopt Kalman filter to inherently mimic the mechanism of SkewIoU. The resulting new loss called KFIoU is easier to implement and works better compared with exact SkewIoU.
  arXiv  Detail & Related papers  (2022-01-29T10:54:57Z)
• SESS: Self-Ensembling Semi-Supervised 3D Object Detection [138.80825169240302]
  We propose SESS, a self-ensembling semi-supervised 3D object detection framework. Specifically, we design a thorough perturbation scheme to enhance generalization of the network on unlabeled and new unseen data. Our SESS achieves competitive performance compared to the state-of-the-art fully-supervised method by using only 50% labeled data.
  arXiv  Detail & Related papers  (2019-12-26T08:48:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.