EdgeGaussians -- 3D Edge Mapping via Gaussian Splatting

• URL: http://arxiv.org/abs/2409.12886v1
• Date: Thu, 19 Sep 2024 16:28:45 GMT
• Title: EdgeGaussians -- 3D Edge Mapping via Gaussian Splatting
• Authors: Kunal Chelani, Assia Benbihi, Torsten Sattler, Fredrik Kahl,
• Abstract summary: State-of-the-art image-based methods learn a 3D edge point cloud then fit 3D edges to it. Our method learns explicitly the 3D edge points and their edge direction hence bypassing the need for point sampling. Results show that the proposed method produces edges as accurate and complete as the state-of-the-art while being an order of magnitude faster.
• Score: 33.43750488033706
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With their meaningful geometry and their omnipresence in the 3D world, edges are extremely useful primitives in computer vision. 3D edges comprise of lines and curves, and methods to reconstruct them use either multi-view images or point clouds as input. State-of-the-art image-based methods first learn a 3D edge point cloud then fit 3D edges to it. The edge point cloud is obtained by learning a 3D neural implicit edge field from which the 3D edge points are sampled on a specific level set (0 or 1). However, such methods present two important drawbacks: i) it is not realistic to sample points on exact level sets due to float imprecision and training inaccuracies. Instead, they are sampled within a range of levels so the points do not lie accurately on the 3D edges and require further processing. ii) Such implicit representations are computationally expensive and require long training times. In this paper, we address these two limitations and propose a 3D edge mapping that is simpler, more efficient, and preserves accuracy. Our method learns explicitly the 3D edge points and their edge direction hence bypassing the need for point sampling. It casts a 3D edge point as the center of a 3D Gaussian and the edge direction as the principal axis of the Gaussian. Such a representation has the advantage of being not only geometrically meaningful but also compatible with the efficient training optimization defined in Gaussian Splatting. Results show that the proposed method produces edges as accurate and complete as the state-of-the-art while being an order of magnitude faster. Code is released at https://github.com/kunalchelani/EdgeGaussians.

Related papers

• OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understanding [54.981605111365056]
  This paper introduces OpenGaussian, a method based on 3D Gaussian Splatting (3DGS) capable of 3D point-level open vocabulary understanding. Our primary motivation stems from observing that existing 3DGS-based open vocabulary methods mainly focus on 2D pixel-level parsing.
  arXiv  Detail & Related papers  (2024-06-04T07:42:33Z)
• 3D Neural Edge Reconstruction [61.10201396044153]
  We introduce EMAP, a new method for learning 3D edge representations with a focus on both lines and curves. Our method implicitly encodes 3D edge distance and direction in Unsigned Distance Functions (UDF) from multi-view edge maps. On top of this neural representation, we propose an edge extraction algorithm that robustly abstracts 3D edges from the inferred edge points and their directions.
  arXiv  Detail & Related papers  (2024-05-29T17:23:51Z)
• EGGS: Edge Guided Gaussian Splatting for Radiance Fields [3.156444853783626]
  We propose an Edge Guided Gaussian Splatting (EGGS) method that leverages the edges in the input images. With such edge guidance, the resulting Gaussian particles focus more on the edges instead of the flat regions. Experiments confirm that such simple edge-weighted loss function indeed improves about $1sim2$ dB on several difference data sets.
  arXiv  Detail & Related papers  (2024-04-14T00:08:56Z)
• SAGD: Boundary-Enhanced Segment Anything in 3D Gaussian via Gaussian Decomposition [66.80822249039235]
  3D Gaussian Splatting has emerged as an alternative 3D representation for novel view synthesis. We propose SAGD, a conceptually simple yet effective boundary-enhanced segmentation pipeline for 3D-GS. Our approach achieves high-quality 3D segmentation without rough boundary issues, which can be easily applied to other scene editing tasks.
  arXiv  Detail & Related papers  (2024-01-31T14:19:03Z)
• NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance [59.08521048003009]
  We propose a neural implicit surface reconstruction pipeline with guidance from 3D Gaussian Splatting to recover highly detailed surfaces. The advantage of 3D Gaussian Splatting is that it can generate dense point clouds with detailed structure. We introduce a scale regularizer to pull the centers close to the surface by enforcing the 3D Gaussians to be extremely thin.
  arXiv  Detail & Related papers  (2023-12-01T07:04:47Z)
• SepicNet: Sharp Edges Recovery by Parametric Inference of Curves in 3D Shapes [16.355677959323426]
  We introduce SepicNet, a novel deep network for the detection and parametrization of sharp edges in 3D shapes as primitive curves. We develop an adaptive point cloud sampling technique that captures the sharp features better than uniform sampling.
  arXiv  Detail & Related papers  (2023-04-13T13:37:21Z)
• NEF: Neural Edge Fields for 3D Parametric Curve Reconstruction from Multi-view Images [18.303674194874457]
  We study the problem of reconstructing 3D feature curves of an object from a set of calibrated multi-view images. We learn a neural implicit field representing the density distribution of 3D edges which we refer to as Neural Edge Field (NEF) NEF is optimized with a view-based rendering loss where a 2D edge map is rendered at a given view and is compared to the ground-truth edge map extracted from the image of that view.
  arXiv  Detail & Related papers  (2023-03-14T06:45:13Z)
• MvDeCor: Multi-view Dense Correspondence Learning for Fine-grained 3D Segmentation [91.6658845016214]
  We propose to utilize self-supervised techniques in the 2D domain for fine-grained 3D shape segmentation tasks. We render a 3D shape from multiple views, and set up a dense correspondence learning task within the contrastive learning framework. As a result, the learned 2D representations are view-invariant and geometrically consistent.
  arXiv  Detail & Related papers  (2022-08-18T00:48:15Z)
• FGR: Frustum-Aware Geometric Reasoning for Weakly Supervised 3D Vehicle Detection [81.79171905308827]
  We propose frustum-aware geometric reasoning (FGR) to detect vehicles in point clouds without any 3D annotations. Our method consists of two stages: coarse 3D segmentation and 3D bounding box estimation. It is able to accurately detect objects in 3D space with only 2D bounding boxes and sparse point clouds.
  arXiv  Detail & Related papers  (2021-05-17T07:29:55Z)
• 3D Shape Segmentation with Geometric Deep Learning [2.512827436728378]
  We propose a neural-network based approach that produces 3D augmented views of the 3D shape to solve the whole segmentation as sub-segmentation problems. We validate our approach using 3D shapes of publicly available datasets and of real objects that are reconstructed using photogrammetry techniques.
  arXiv  Detail & Related papers  (2020-02-02T14:11:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.