Multi-interactive Dual-decoder for RGB-thermal Salient Object Detection

• URL: http://arxiv.org/abs/2005.02315v3
• Date: Fri, 4 Jun 2021 12:56:34 GMT
• Title: Multi-interactive Dual-decoder for RGB-thermal Salient Object Detection
• Authors: Zhengzheng Tu, Zhun Li, Chenglong Li, Yang Lang, Jin Tang
• Abstract summary: RGB-thermal salient object detection (SOD) aims to segment the common prominent regions of visible image and corresponding thermal infrared image. Existing methods don't fully explore and exploit the potentials of complementarity of different modalities and multi-type cues of image contents. We propose a multi-interactive dual-decoder to mine and model the multi-type interactions for accurate RGBT SOD.
• Score: 37.79290349045164
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: RGB-thermal salient object detection (SOD) aims to segment the common prominent regions of visible image and corresponding thermal infrared image that we call it RGBT SOD. Existing methods don't fully explore and exploit the potentials of complementarity of different modalities and multi-type cues of image contents, which play a vital role in achieving accurate results. In this paper, we propose a multi-interactive dual-decoder to mine and model the multi-type interactions for accurate RGBT SOD. In specific, we first encode two modalities into multi-level multi-modal feature representations. Then, we design a novel dual-decoder to conduct the interactions of multi-level features, two modalities and global contexts. With these interactions, our method works well in diversely challenging scenarios even in the presence of invalid modality. Finally, we carry out extensive experiments on public RGBT and RGBD SOD datasets, and the results show that the proposed method achieves the outstanding performance against state-of-the-art algorithms. The source code has been released at:https://github.com/lz118/Multi-interactive-Dual-decoder.

Related papers

• HODINet: High-Order Discrepant Interaction Network for RGB-D Salient Object Detection [4.007827908611563]
  RGB-D salient object detection (SOD) aims to detect the prominent regions by jointly modeling RGB and depth information. Most RGB-D SOD methods apply the same type of backbones and fusion modules to identically learn the multimodality and multistage features. In this paper, we propose a high-order discrepant interaction network (HODINet) for RGB-D SOD.
  arXiv  Detail & Related papers  (2023-07-03T11:56:21Z)
• Multimodal Industrial Anomaly Detection via Hybrid Fusion [59.16333340582885]
  We propose a novel multimodal anomaly detection method with hybrid fusion scheme. Our model outperforms the state-of-the-art (SOTA) methods on both detection and segmentation precision on MVTecD-3 AD dataset.
  arXiv  Detail & Related papers  (2023-03-01T15:48:27Z)
• HiDAnet: RGB-D Salient Object Detection via Hierarchical Depth Awareness [2.341385717236931]
  We propose a novel Hierarchical Depth Awareness network (HiDAnet) for RGB-D saliency detection. Our motivation comes from the observation that the multi-granularity properties of geometric priors correlate well with the neural network hierarchies. Our HiDAnet performs favorably over the state-of-the-art methods by large margins.
  arXiv  Detail & Related papers  (2023-01-18T10:00:59Z)
• Interactive Context-Aware Network for RGB-T Salient Object Detection [7.544240329265388]
  We propose a novel network called Interactive Context-Aware Network (ICANet) ICANet contains three modules that can effectively perform the cross-modal and cross-scale fusions. Experiments prove that our network performs favorably against the state-of-the-art RGB-T SOD methods.
  arXiv  Detail & Related papers  (2022-11-11T10:04:36Z)
• Dual Swin-Transformer based Mutual Interactive Network for RGB-D Salient Object Detection [67.33924278729903]
  In this work, we propose Dual Swin-Transformer based Mutual Interactive Network. We adopt Swin-Transformer as the feature extractor for both RGB and depth modality to model the long-range dependencies in visual inputs. Comprehensive experiments on five standard RGB-D SOD benchmark datasets demonstrate the superiority of the proposed DTMINet method.
  arXiv  Detail & Related papers  (2022-06-07T08:35:41Z)
• Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection [65.30079184700755]
  This study addresses the issue of fusing infrared and visible images that appear differently for object detection. Previous approaches discover commons underlying the two modalities and fuse upon the common space either by iterative optimization or deep networks. This paper proposes a bilevel optimization formulation for the joint problem of fusion and detection, and then unrolls to a target-aware Dual Adversarial Learning (TarDAL) network for fusion and a commonly used detection network.
  arXiv  Detail & Related papers  (2022-03-30T11:44:56Z)
• Multi-Scale Iterative Refinement Network for RGB-D Salient Object Detection [7.062058947498447]
  salient visual cues appear in various scales and resolutions of RGB images due to semantic gaps at different feature levels. Similar salient patterns are available in cross-modal depth images as well as multi-scale versions. We devise attention based fusion module (ABF) to address on cross-modal correlation.
  arXiv  Detail & Related papers  (2022-01-24T10:33:00Z)
• Transformer-based Network for RGB-D Saliency Detection [82.6665619584628]
  Key to RGB-D saliency detection is to fully mine and fuse information at multiple scales across the two modalities. We show that transformer is a uniform operation which presents great efficacy in both feature fusion and feature enhancement. Our proposed network performs favorably against state-of-the-art RGB-D saliency detection methods.
  arXiv  Detail & Related papers  (2021-12-01T15:53:58Z)
• M2RNet: Multi-modal and Multi-scale Refined Network for RGB-D Salient Object Detection [1.002712867721496]
  Methods based on RGB-D often suffer from the incompatibility of multi-modal feature fusion and the insufficiency of multi-scale feature aggregation. We propose a novel multi-modal and multi-scale refined network (M2RNet) Three essential components are presented in this network.
  arXiv  Detail & Related papers  (2021-09-16T12:15:40Z)
• Cross-modality Discrepant Interaction Network for RGB-D Salient Object Detection [78.47767202232298]
  We propose a novel Cross-modality Discrepant Interaction Network (CDINet) for RGB-D SOD. Two components are designed to implement the effective cross-modality interaction. Our network outperforms $15$ state-of-the-art methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2021-08-04T11:24:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.