Interactive Context-Aware Network for RGB-T Salient Object Detection

• URL: http://arxiv.org/abs/2211.06097v1
• Date: Fri, 11 Nov 2022 10:04:36 GMT
• Title: Interactive Context-Aware Network for RGB-T Salient Object Detection
• Authors: Yuxuan Wang, Feng Dong, Jinchao Zhu
• Abstract summary: We propose a novel network called Interactive Context-Aware Network (ICANet) ICANet contains three modules that can effectively perform the cross-modal and cross-scale fusions. Experiments prove that our network performs favorably against the state-of-the-art RGB-T SOD methods.
• Score: 7.544240329265388
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Salient object detection (SOD) focuses on distinguishing the most conspicuous objects in the scene. However, most related works are based on RGB images, which lose massive useful information. Accordingly, with the maturity of thermal technology, RGB-T (RGB-Thermal) multi-modality tasks attain more and more attention. Thermal infrared images carry important information which can be used to improve the accuracy of SOD prediction. To accomplish it, the methods to integrate multi-modal information and suppress noises are critical. In this paper, we propose a novel network called Interactive Context-Aware Network (ICANet). It contains three modules that can effectively perform the cross-modal and cross-scale fusions. We design a Hybrid Feature Fusion (HFF) module to integrate the features of two modalities, which utilizes two types of feature extraction. The Multi-Scale Attention Reinforcement (MSAR) and Upper Fusion (UF) blocks are responsible for the cross-scale fusion that converges different levels of features and generate the prediction maps. We also raise a novel Context-Aware Multi-Supervised Network (CAMSNet) to calculate the content loss between the prediction and the ground truth (GT). Experiments prove that our network performs favorably against the state-of-the-art RGB-T SOD methods.

Related papers

• HODINet: High-Order Discrepant Interaction Network for RGB-D Salient Object Detection [4.007827908611563]
  RGB-D salient object detection (SOD) aims to detect the prominent regions by jointly modeling RGB and depth information. Most RGB-D SOD methods apply the same type of backbones and fusion modules to identically learn the multimodality and multistage features. In this paper, we propose a high-order discrepant interaction network (HODINet) for RGB-D SOD.
  arXiv  Detail & Related papers  (2023-07-03T11:56:21Z)
• Dual Swin-Transformer based Mutual Interactive Network for RGB-D Salient Object Detection [67.33924278729903]
  In this work, we propose Dual Swin-Transformer based Mutual Interactive Network. We adopt Swin-Transformer as the feature extractor for both RGB and depth modality to model the long-range dependencies in visual inputs. Comprehensive experiments on five standard RGB-D SOD benchmark datasets demonstrate the superiority of the proposed DTMINet method.
  arXiv  Detail & Related papers  (2022-06-07T08:35:41Z)
• Multi-Scale Iterative Refinement Network for RGB-D Salient Object Detection [7.062058947498447]
  salient visual cues appear in various scales and resolutions of RGB images due to semantic gaps at different feature levels. Similar salient patterns are available in cross-modal depth images as well as multi-scale versions. We devise attention based fusion module (ABF) to address on cross-modal correlation.
  arXiv  Detail & Related papers  (2022-01-24T10:33:00Z)
• Transformer-based Network for RGB-D Saliency Detection [82.6665619584628]
  Key to RGB-D saliency detection is to fully mine and fuse information at multiple scales across the two modalities. We show that transformer is a uniform operation which presents great efficacy in both feature fusion and feature enhancement. Our proposed network performs favorably against state-of-the-art RGB-D saliency detection methods.
  arXiv  Detail & Related papers  (2021-12-01T15:53:58Z)
• Modality-Guided Subnetwork for Salient Object Detection [5.491692465987937]
  Most RGBD networks require multi-modalities from the input side and feed them separately through a two-stream design. We present in this paper a novel fusion design named modality-guided subnetwork (MGSnet) It has the following superior designs: 1) Our model works for both RGB and RGBD data, and dynamically estimating depth if not available.
  arXiv  Detail & Related papers  (2021-10-10T20:59:11Z)
• M2RNet: Multi-modal and Multi-scale Refined Network for RGB-D Salient Object Detection [1.002712867721496]
  Methods based on RGB-D often suffer from the incompatibility of multi-modal feature fusion and the insufficiency of multi-scale feature aggregation. We propose a novel multi-modal and multi-scale refined network (M2RNet) Three essential components are presented in this network.
  arXiv  Detail & Related papers  (2021-09-16T12:15:40Z)
• Self-Supervised Representation Learning for RGB-D Salient Object Detection [93.17479956795862]
  We use Self-Supervised Representation Learning to design two pretext tasks: the cross-modal auto-encoder and the depth-contour estimation. Our pretext tasks require only a few and un RGB-D datasets to perform pre-training, which make the network capture rich semantic contexts. For the inherent problem of cross-modal fusion in RGB-D SOD, we propose a multi-path fusion module.
  arXiv  Detail & Related papers  (2021-01-29T09:16:06Z)
• Siamese Network for RGB-D Salient Object Detection and Beyond [113.30063105890041]
  A novel framework is proposed to learn from both RGB and depth inputs through a shared network backbone. Comprehensive experiments using five popular metrics show that the designed framework yields a robust RGB-D saliency detector. We also link JL-DCF to the RGB-D semantic segmentation field, showing its capability of outperforming several semantic segmentation models.
  arXiv  Detail & Related papers  (2020-08-26T06:01:05Z)
• RGB-D Salient Object Detection with Cross-Modality Modulation and Selection [126.4462739820643]
  We present an effective method to progressively integrate and refine the cross-modality complementarities for RGB-D salient object detection (SOD) The proposed network mainly solves two challenging issues: 1) how to effectively integrate the complementary information from RGB image and its corresponding depth map, and 2) how to adaptively select more saliency-related features.
  arXiv  Detail & Related papers  (2020-07-14T14:22:50Z)
• Multi-level Cross-modal Interaction Network for RGB-D Salient Object Detection [3.581367375462018]
  We propose a novel Multi-level Cross-modal Interaction Network (MCINet) for RGB-D based salient object detection (SOD) Our MCI-Net includes two key components: 1) a cross-modal feature learning network, which is used to learn the high-level features for the RGB images and depth cues, effectively enabling the correlations between the two sources to be exploited; and 2) a multi-level interactive integration network, which integrates multi-level cross-modal features to boost the SOD performance.
  arXiv  Detail & Related papers  (2020-07-10T02:21:02Z)
• Cross-Modal Weighting Network for RGB-D Salient Object Detection [76.0965123893641]
  We propose a novel Cross-Modal Weighting (CMW) strategy to encourage comprehensive interactions between RGB and depth channels for RGB-D SOD. Specifically, three RGB-depth interaction modules, named CMW-L, CMW-M and CMW-H, are developed to deal with respectively low-, middle- and high-level cross-modal information fusion. CMWNet consistently outperforms 15 state-of-the-art RGB-D SOD methods on seven popular benchmarks.
  arXiv  Detail & Related papers  (2020-07-09T16:01:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.