Cross-Modal Weighting Network for RGB-D Salient Object Detection
- URL: http://arxiv.org/abs/2007.04901v1
- Date: Thu, 9 Jul 2020 16:01:44 GMT
- Title: Cross-Modal Weighting Network for RGB-D Salient Object Detection
- Authors: Gongyang Li, Zhi Liu, Linwei Ye, Yang Wang, Haibin Ling
- Abstract summary: We propose a novel Cross-Modal Weighting (CMW) strategy to encourage comprehensive interactions between RGB and depth channels for RGB-D SOD.
Specifically, three RGB-depth interaction modules, named CMW-L, CMW-M and CMW-H, are developed to deal with respectively low-, middle- and high-level cross-modal information fusion.
CMWNet consistently outperforms 15 state-of-the-art RGB-D SOD methods on seven popular benchmarks.
- Score: 76.0965123893641
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Depth maps contain geometric clues for assisting Salient Object Detection
(SOD). In this paper, we propose a novel Cross-Modal Weighting (CMW) strategy
to encourage comprehensive interactions between RGB and depth channels for
RGB-D SOD. Specifically, three RGB-depth interaction modules, named CMW-L,
CMW-M and CMW-H, are developed to deal with respectively low-, middle- and
high-level cross-modal information fusion. These modules use Depth-to-RGB
Weighing (DW) and RGB-to-RGB Weighting (RW) to allow rich cross-modal and
cross-scale interactions among feature layers generated by different network
blocks. To effectively train the proposed Cross-Modal Weighting Network
(CMWNet), we design a composite loss function that summarizes the errors
between intermediate predictions and ground truth over different scales. With
all these novel components working together, CMWNet effectively fuses
information from RGB and depth channels, and meanwhile explores object
localization and details across scales. Thorough evaluations demonstrate CMWNet
consistently outperforms 15 state-of-the-art RGB-D SOD methods on seven popular
benchmarks.
Related papers
- HODINet: High-Order Discrepant Interaction Network for RGB-D Salient
Object Detection [4.007827908611563]
RGB-D salient object detection (SOD) aims to detect the prominent regions by jointly modeling RGB and depth information.
Most RGB-D SOD methods apply the same type of backbones and fusion modules to identically learn the multimodality and multistage features.
In this paper, we propose a high-order discrepant interaction network (HODINet) for RGB-D SOD.
arXiv Detail & Related papers (2023-07-03T11:56:21Z) - Interactive Context-Aware Network for RGB-T Salient Object Detection [7.544240329265388]
We propose a novel network called Interactive Context-Aware Network (ICANet)
ICANet contains three modules that can effectively perform the cross-modal and cross-scale fusions.
Experiments prove that our network performs favorably against the state-of-the-art RGB-T SOD methods.
arXiv Detail & Related papers (2022-11-11T10:04:36Z) - CIR-Net: Cross-modality Interaction and Refinement for RGB-D Salient
Object Detection [144.66411561224507]
We present a convolutional neural network (CNN) model, named CIR-Net, based on the novel cross-modality interaction and refinement.
Our network outperforms the state-of-the-art saliency detectors both qualitatively and quantitatively.
arXiv Detail & Related papers (2022-10-06T11:59:19Z) - Modality-Guided Subnetwork for Salient Object Detection [5.491692465987937]
Most RGBD networks require multi-modalities from the input side and feed them separately through a two-stream design.
We present in this paper a novel fusion design named modality-guided subnetwork (MGSnet)
It has the following superior designs: 1) Our model works for both RGB and RGBD data, and dynamically estimating depth if not available.
arXiv Detail & Related papers (2021-10-10T20:59:11Z) - Cross-modality Discrepant Interaction Network for RGB-D Salient Object
Detection [78.47767202232298]
We propose a novel Cross-modality Discrepant Interaction Network (CDINet) for RGB-D SOD.
Two components are designed to implement the effective cross-modality interaction.
Our network outperforms $15$ state-of-the-art methods both quantitatively and qualitatively.
arXiv Detail & Related papers (2021-08-04T11:24:42Z) - Siamese Network for RGB-D Salient Object Detection and Beyond [113.30063105890041]
A novel framework is proposed to learn from both RGB and depth inputs through a shared network backbone.
Comprehensive experiments using five popular metrics show that the designed framework yields a robust RGB-D saliency detector.
We also link JL-DCF to the RGB-D semantic segmentation field, showing its capability of outperforming several semantic segmentation models.
arXiv Detail & Related papers (2020-08-26T06:01:05Z) - Data-Level Recombination and Lightweight Fusion Scheme for RGB-D Salient
Object Detection [73.31632581915201]
We propose a novel data-level recombination strategy to fuse RGB with D (depth) before deep feature extraction.
A newly lightweight designed triple-stream network is applied over these novel formulated data to achieve an optimal channel-wise complementary fusion status between the RGB and D.
arXiv Detail & Related papers (2020-08-07T10:13:05Z) - Bi-directional Cross-Modality Feature Propagation with
Separation-and-Aggregation Gate for RGB-D Semantic Segmentation [59.94819184452694]
Depth information has proven to be a useful cue in the semantic segmentation of RGBD images for providing a geometric counterpart to the RGB representation.
Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion.
In this paper, we propose a unified and efficient Crossmodality Guided to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
arXiv Detail & Related papers (2020-07-17T18:35:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.