On the Importance of Accurate Geometry Data for Dense 3D Vision Tasks
- URL: http://arxiv.org/abs/2303.14840v1
- Date: Sun, 26 Mar 2023 22:32:44 GMT
- Title: On the Importance of Accurate Geometry Data for Dense 3D Vision Tasks
- Authors: HyunJun Jung, Patrick Ruhkamp, Guangyao Zhai, Nikolas Brasch, Yitong
Li, Yannick Verdie, Jifei Song, Yiren Zhou, Anil Armagan, Slobodan Ilic, Ales
Leonardis, Nassir Navab, Benjamin Busam
- Abstract summary: Training on inaccurate or corrupt data induces model bias and hampers generalisation capabilities.
This paper investigates the effect of sensor errors for the dense 3D vision tasks of depth estimation and reconstruction.
- Score: 61.74608497496841
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Learning-based methods to solve dense 3D vision problems typically train on
3D sensor data. The respectively used principle of measuring distances provides
advantages and drawbacks. These are typically not compared nor discussed in the
literature due to a lack of multi-modal datasets. Texture-less regions are
problematic for structure from motion and stereo, reflective material poses
issues for active sensing, and distances for translucent objects are intricate
to measure with existing hardware. Training on inaccurate or corrupt data
induces model bias and hampers generalisation capabilities. These effects
remain unnoticed if the sensor measurement is considered as ground truth during
the evaluation. This paper investigates the effect of sensor errors for the
dense 3D vision tasks of depth estimation and reconstruction. We rigorously
show the significant impact of sensor characteristics on the learned
predictions and notice generalisation issues arising from various technologies
in everyday household environments. For evaluation, we introduce a carefully
designed dataset\footnote{dataset available at
https://github.com/Junggy/HAMMER-dataset} comprising measurements from
commodity sensors, namely D-ToF, I-ToF, passive/active stereo, and monocular
RGB+P. Our study quantifies the considerable sensor noise impact and paves the
way to improved dense vision estimates and targeted data fusion.
Related papers
- CatFree3D: Category-agnostic 3D Object Detection with Diffusion [63.75470913278591]
We introduce a novel pipeline that decouples 3D detection from 2D detection and depth prediction.
We also introduce the Normalised Hungarian Distance (NHD) metric for an accurate evaluation of 3D detection results.
arXiv Detail & Related papers (2024-08-22T22:05:57Z) - PruNeRF: Segment-Centric Dataset Pruning via 3D Spatial Consistency [33.68948881727943]
PruNeRF is a segment-centric dataset pruning framework via 3D spatial consistency.
Our experiments on benchmark datasets demonstrate that PruNeRF consistently outperforms state-of-the-art methods in robustness against distractors.
arXiv Detail & Related papers (2024-06-02T16:49:05Z) - SimMining-3D: Altitude-Aware 3D Object Detection in Complex Mining
Environments: A Novel Dataset and ROS-Based Automatic Annotation Pipeline [0.9790236766474201]
We introduce a synthetic dataset SimMining 3D specifically designed for 3D object detection in mining environments.
The dataset captures objects and sensors positioned at various heights within mine benches, accurately reflecting authentic mining scenarios.
We propose evaluation metrics accounting for sensor-to-object height variations and point cloud density, enabling accurate model assessment.
arXiv Detail & Related papers (2023-12-11T04:33:45Z) - Towards a Robust Sensor Fusion Step for 3D Object Detection on Corrupted
Data [4.3012765978447565]
This work presents a novel fusion step that addresses data corruptions and makes sensor fusion for 3D object detection more robust.
We demonstrate that our method performs on par with state-of-the-art approaches on normal data and outperforms them on misaligned data.
arXiv Detail & Related papers (2023-06-12T18:06:29Z) - AGO-Net: Association-Guided 3D Point Cloud Object Detection Network [86.10213302724085]
We propose a novel 3D detection framework that associates intact features for objects via domain adaptation.
We achieve new state-of-the-art performance on the KITTI 3D detection benchmark in both accuracy and speed.
arXiv Detail & Related papers (2022-08-24T16:54:38Z) - Is my Depth Ground-Truth Good Enough? HAMMER -- Highly Accurate
Multi-Modal Dataset for DEnse 3D Scene Regression [34.95597838973912]
HAMMER is a dataset comprising depth estimates from multiple commonly used sensors for indoor depth estimation.
We construct highly reliable ground truth depth maps with the help of 3D scanners and aligned renderings.
A popular depth estimators is trained on this data and typical depth senosors.
arXiv Detail & Related papers (2022-05-09T21:25:09Z) - Probabilistic and Geometric Depth: Detecting Objects in Perspective [78.00922683083776]
3D object detection is an important capability needed in various practical applications such as driver assistance systems.
Monocular 3D detection, as an economical solution compared to conventional settings relying on binocular vision or LiDAR, has drawn increasing attention recently but still yields unsatisfactory results.
This paper first presents a systematic study on this problem and observes that the current monocular 3D detection problem can be simplified as an instance depth estimation problem.
arXiv Detail & Related papers (2021-07-29T16:30:33Z) - 3D Object Detection for Autonomous Driving: A Survey [14.772968858398043]
3D object detection serves as the core basis of such perception system.
Despite existing efforts, 3D object detection on point clouds is still in its infancy.
Recent state-of-the-art detection methods with their pros and cons are presented.
arXiv Detail & Related papers (2021-06-21T03:17:20Z) - Delving into Localization Errors for Monocular 3D Object Detection [85.77319416168362]
Estimating 3D bounding boxes from monocular images is an essential component in autonomous driving.
In this work, we quantify the impact introduced by each sub-task and find the localization error' is the vital factor in restricting monocular 3D detection.
arXiv Detail & Related papers (2021-03-30T10:38:01Z) - SESS: Self-Ensembling Semi-Supervised 3D Object Detection [138.80825169240302]
We propose SESS, a self-ensembling semi-supervised 3D object detection framework. Specifically, we design a thorough perturbation scheme to enhance generalization of the network on unlabeled and new unseen data.
Our SESS achieves competitive performance compared to the state-of-the-art fully-supervised method by using only 50% labeled data.
arXiv Detail & Related papers (2019-12-26T08:48:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.