Learning Pseudo Front Depth for 2D Forward-Looking Sonar-based Multi-view Stereo

• URL: http://arxiv.org/abs/2208.00233v1
• Date: Sat, 30 Jul 2022 14:35:21 GMT
• Title: Learning Pseudo Front Depth for 2D Forward-Looking Sonar-based Multi-view Stereo
• Authors: Yusheng Wang and Yonghoon Ji and Hiroshi Tsuchiya and Hajime Asama and Atsushi Yamashita
• Abstract summary: Retrieving the missing dimension information in acoustic images from 2D forward-looking sonar is a well-known problem in the field of underwater robotics. We propose a novel learning-based multi-view stereo method to estimate 3D information.
• Score: 5.024813922014977
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieving the missing dimension information in acoustic images from 2D forward-looking sonar is a well-known problem in the field of underwater robotics. There are works attempting to retrieve 3D information from a single image which allows the robot to generate 3D maps with fly-through motion. However, owing to the unique image formulation principle, estimating 3D information from a single image faces severe ambiguity problems. Classical methods of multi-view stereo can avoid the ambiguity problems, but may require a large number of viewpoints to generate an accurate model. In this work, we propose a novel learning-based multi-view stereo method to estimate 3D information. To better utilize the information from multiple frames, an elevation plane sweeping method is proposed to generate the depth-azimuth-elevation cost volume. The volume after regularization can be considered as a probabilistic volumetric representation of the target. Instead of performing regression on the elevation angles, we use pseudo front depth from the cost volume to represent the 3D information which can avoid the 2D-3D problem in acoustic imaging. High-accuracy results can be generated with only two or three images. Synthetic datasets were generated to simulate various underwater targets. We also built the first real dataset with accurate ground truth in a large scale water tank. Experimental results demonstrate the superiority of our method, compared to other state-of-the-art methods.

Related papers

• Object Modeling from Underwater Forward-Scan Sonar Imagery with Sea-Surface Multipath [16.057203527513632]
  A key contribution, for objects imaged in the proximity of the sea surface, is to resolve the multipath artifacts due to the air-water interface. Here, the object image formed by the direct target backscatter is almost always corrupted by the ghost and sometimes by the mirror components. We model, localize, and discard the corrupted object region within each view, thus avoiding the distortion of recovered 3-D shape.
  arXiv  Detail & Related papers  (2024-09-10T18:46:25Z)
• Creating Seamless 3D Maps Using Radiance Fields [0.0]
  It is desirable to create 3D object models and 3D maps from 2D input images for applications such as navigation, virtual tourism, and urban planning. Traditional methods have difficulty with reflective surfaces and specular reflections. Google Road View does not create an actual 3D object model, only a collection of views.
  arXiv  Detail & Related papers  (2024-03-17T22:49:07Z)
• PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm [114.47216525866435]
  We introduce a novel universal 3D pre-training framework designed to facilitate the acquisition of efficient 3D representation. For the first time, PonderV2 achieves state-of-the-art performance on 11 indoor and outdoor benchmarks, implying its effectiveness.
  arXiv  Detail & Related papers  (2023-10-12T17:59:57Z)
• OccDepth: A Depth-Aware Method for 3D Semantic Scene Completion [6.297023466646343]
  3D Semantic Scene Completion (SSC) can provide dense geometric and semantic scene representations, which can be applied in the field of autonomous driving and robotic systems. We propose the first stereo SSC method named OccDepth, which fully exploits implicit depth information from stereo images (or RGBD images) to help the recovery of 3D geometric structures.
  arXiv  Detail & Related papers  (2023-02-27T06:35:03Z)
• Time and Cost-Efficient Bathymetric Mapping System using Sparse Point Cloud Generation and Automatic Object Detection [0.0]
  Side-scan sonar sensors are available in inexpensive cost ranges, especially in fish-finders. Extracting 3D information from side-scan sonar imagery is a difficult task because of its low signal-to-noise ratio. This paper introduces an efficient algorithm that generates a sparse 3D point cloud from side-scan sonar images.
  arXiv  Detail & Related papers  (2022-10-19T02:58:08Z)
• Monocular 3D Object Detection with Depth from Motion [74.29588921594853]
  We take advantage of camera ego-motion for accurate object depth estimation and detection. Our framework, named Depth from Motion (DfM), then uses the established geometry to lift 2D image features to the 3D space and detects 3D objects thereon. Our framework outperforms state-of-the-art methods by a large margin on the KITTI benchmark.
  arXiv  Detail & Related papers  (2022-07-26T15:48:46Z)
• DSGN++: Exploiting Visual-Spatial Relation forStereo-based 3D Detectors [60.88824519770208]
  Camera-based 3D object detectors are welcome due to their wider deployment and lower price than LiDAR sensors. We revisit the prior stereo modeling DSGN about the stereo volume constructions for representing both 3D geometry and semantics. We propose our approach, DSGN++, aiming for improving information flow throughout the 2D-to-3D pipeline.
  arXiv  Detail & Related papers  (2022-04-06T18:43:54Z)
• DRaCoN -- Differentiable Rasterization Conditioned Neural Radiance Fields for Articulated Avatars [92.37436369781692]
  We present DRaCoN, a framework for learning full-body volumetric avatars. It exploits the advantages of both the 2D and 3D neural rendering techniques. Experiments on the challenging ZJU-MoCap and Human3.6M datasets indicate that DRaCoN outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-03-29T17:59:15Z)
• Improved Modeling of 3D Shapes with Multi-view Depth Maps [48.8309897766904]
  We present a general-purpose framework for modeling 3D shapes using CNNs. Using just a single depth image of the object, we can output a dense multi-view depth map representation of 3D objects.
  arXiv  Detail & Related papers  (2020-09-07T17:58:27Z)
• HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map [72.93634777578336]
  We propose a novel architecture with 3D convolutions trained in a weakly-supervised manner. The proposed approach improves over the state of the art by 47.8% on the SynHand5M dataset. Our method produces visually more reasonable and realistic hand shapes on NYU and BigHand2.2M datasets.
  arXiv  Detail & Related papers  (2020-04-03T14:27:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.