Adaptive-Mask Fusion Network for Segmentation of Drivable Road and Negative Obstacle With Untrustworthy Features

• URL: http://arxiv.org/abs/2304.13979v1
• Date: Thu, 27 Apr 2023 07:00:45 GMT
• Title: Adaptive-Mask Fusion Network for Segmentation of Drivable Road and Negative Obstacle With Untrustworthy Features
• Authors: Zhen Feng, Yuchao Feng, Yanning Guo, Yuxiang Sun
• Abstract summary: We find that when fusing two modals of data with untrustworthy features, the performance of multi-modal networks could be degraded. We propose the Adaptive-Mask Fusion Network (AMFNet) by adaptive-weight masks in the fusion module to fuse features from RGB and depth images with inconsistency. Our network achieves state-of-the-art performance compared with other networks.
• Score: 7.6830358429947
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Segmentation of drivable roads and negative obstacles is critical to the safe driving of autonomous vehicles. Currently, many multi-modal fusion methods have been proposed to improve segmentation accuracy, such as fusing RGB and depth images. However, we find that when fusing two modals of data with untrustworthy features, the performance of multi-modal networks could be degraded, even lower than those using a single modality. In this paper, the untrustworthy features refer to those extracted from regions (e.g., far objects that are beyond the depth measurement range) with invalid depth data (i.e., 0 pixel value) in depth images. The untrustworthy features can confuse the segmentation results, and hence lead to inferior results. To provide a solution to this issue, we propose the Adaptive-Mask Fusion Network (AMFNet) by introducing adaptive-weight masks in the fusion module to fuse features from RGB and depth images with inconsistency. In addition, we release a large-scale RGB-depth dataset with manually-labeled ground truth based on the NPO dataset for drivable roads and negative obstacles segmentation. Extensive experimental results demonstrate that our network achieves state-of-the-art performance compared with other networks. Our code and dataset are available at: https://github.com/lab-sun/AMFNet.

Related papers

• Mutual-Guided Dynamic Network for Image Fusion [51.615598671899335]
  We propose a novel mutual-guided dynamic network (MGDN) for image fusion, which allows for effective information utilization across different locations and inputs. Experimental results on five benchmark datasets demonstrate that our proposed method outperforms existing methods on four image fusion tasks.
  arXiv  Detail & Related papers  (2023-08-24T03:50:37Z)
• Pyramid Deep Fusion Network for Two-Hand Reconstruction from RGB-D Images [11.100398985633754]
  We propose an end-to-end framework for recovering dense meshes for both hands. Our framework employs ResNet50 and PointNet++ to derive features from RGB and point cloud. We also introduce a novel pyramid deep fusion network (PDFNet) to aggregate features at different scales.
  arXiv  Detail & Related papers  (2023-07-12T09:33:21Z)
• Scale Invariant Semantic Segmentation with RGB-D Fusion [12.650574326251023]
  We propose a neural network architecture for scale-invariant semantic segmentation using RGB-D images. We incorporate depth information to the RGB data for pixel-wise semantic segmentation to address the different scale objects in an outdoor scene. Our model is compact and can be easily applied to the other RGB model.
  arXiv  Detail & Related papers  (2022-04-10T12:54:27Z)
• Fast Road Segmentation via Uncertainty-aware Symmetric Network [15.05244258071472]
  Prior methods cannot achieve high inference speed and high accuracy in both ways. The different properties of RGB and depth data are not well-exploited, limiting the reliability of predicted road. We propose an uncertainty-aware symmetric network (USNet) to achieve a trade-off between speed and accuracy by fully fusing RGB and depth data.
  arXiv  Detail & Related papers  (2022-03-09T06:11:29Z)
• MTFNet: Mutual-Transformer Fusion Network for RGB-D Salient Object Detection [15.371153771528093]
  We propose a novel Mutual-Transformer Fusion Network (MTFNet) for RGB-D SOD. MTFNet contains two main modules, $i.e.$, Focal Feature Extractor (FFE) and Mutual-Transformer Fusion (MTF) Comprehensive experimental results on six public benchmarks demonstrate the superiority of our proposed MTFNet.
  arXiv  Detail & Related papers  (2021-12-02T12:48:37Z)
• EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation [62.210091681352914]
  We study multi-sensor fusion for 3D semantic segmentation for many applications, such as autonomous driving and robotics. In this work, we investigate a collaborative fusion scheme called perception-aware multi-sensor fusion (PMF) We propose a two-stream network to extract features from the two modalities separately. The extracted features are fused by effective residual-based fusion modules.
  arXiv  Detail & Related papers  (2021-06-21T10:47:26Z)
• Self-Supervised Representation Learning for RGB-D Salient Object Detection [93.17479956795862]
  We use Self-Supervised Representation Learning to design two pretext tasks: the cross-modal auto-encoder and the depth-contour estimation. Our pretext tasks require only a few and un RGB-D datasets to perform pre-training, which make the network capture rich semantic contexts. For the inherent problem of cross-modal fusion in RGB-D SOD, we propose a multi-path fusion module.
  arXiv  Detail & Related papers  (2021-01-29T09:16:06Z)
• MobileSal: Extremely Efficient RGB-D Salient Object Detection [62.04876251927581]
  This paper introduces a novel network, methodname, which focuses on efficient RGB-D salient object detection (SOD) We propose an implicit depth restoration (IDR) technique to strengthen the feature representation capability of mobile networks for RGB-D SOD. With IDR and CPR incorporated, methodnameperforms favorably against sArt methods on seven challenging RGB-D SOD datasets.
  arXiv  Detail & Related papers  (2020-12-24T04:36:42Z)
• Progressively Guided Alternate Refinement Network for RGB-D Salient Object Detection [63.18846475183332]
  We aim to develop an efficient and compact deep network for RGB-D salient object detection. We propose a progressively guided alternate refinement network to refine it. Our model outperforms existing state-of-the-art approaches by a large margin.
  arXiv  Detail & Related papers  (2020-08-17T02:55:06Z)
• Accurate RGB-D Salient Object Detection via Collaborative Learning [101.82654054191443]
  RGB-D saliency detection shows impressive ability on some challenge scenarios. We propose a novel collaborative learning framework where edge, depth and saliency are leveraged in a more efficient way.
  arXiv  Detail & Related papers  (2020-07-23T04:33:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.