Boosting RGB-D Saliency Detection by Leveraging Unlabeled RGB Images
- URL: http://arxiv.org/abs/2201.00100v1
- Date: Sat, 1 Jan 2022 03:02:27 GMT
- Title: Boosting RGB-D Saliency Detection by Leveraging Unlabeled RGB Images
- Authors: Xiaoqiang Wang, Lei Zhu, Siliang Tang, Huazhu Fu, Ping Li, Fei Wu, Yi
Yang, Yueting Zhuang
- Abstract summary: Training deep models for RGB-D salient object detection (SOD) often requires a large number of labeled RGB-D images.
We present a Dual-Semi RGB-D Salient Object Detection Network (DS-Net) to leverage unlabeled RGB images for boosting RGB-D saliency detection.
- Score: 89.81919625224103
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Training deep models for RGB-D salient object detection (SOD) often requires
a large number of labeled RGB-D images. However, RGB-D data is not easily
acquired, which limits the development of RGB-D SOD techniques. To alleviate
this issue, we present a Dual-Semi RGB-D Salient Object Detection Network
(DS-Net) to leverage unlabeled RGB images for boosting RGB-D saliency
detection. We first devise a depth decoupling convolutional neural network
(DDCNN), which contains a depth estimation branch and a saliency detection
branch. The depth estimation branch is trained with RGB-D images and then used
to estimate the pseudo depth maps for all unlabeled RGB images to form the
paired data. The saliency detection branch is used to fuse the RGB feature and
depth feature to predict the RGB-D saliency. Then, the whole DDCNN is assigned
as the backbone in a teacher-student framework for semi-supervised learning.
Moreover, we also introduce a consistency loss on the intermediate attention
and saliency maps for the unlabeled data, as well as a supervised depth and
saliency loss for labeled data. Experimental results on seven widely-used
benchmark datasets demonstrate that our DDCNN outperforms state-of-the-art
methods both quantitatively and qualitatively. We also demonstrate that our
semi-supervised DS-Net can further improve the performance, even when using an
RGB image with the pseudo depth map.
Related papers
- DFormer: Rethinking RGBD Representation Learning for Semantic
Segmentation [76.81628995237058]
DFormer is a novel framework to learn transferable representations for RGB-D segmentation tasks.
It pretrains the backbone using image-depth pairs from ImageNet-1K.
DFormer achieves new state-of-the-art performance on two popular RGB-D tasks.
arXiv Detail & Related papers (2023-09-18T11:09:11Z) - SiaTrans: Siamese Transformer Network for RGB-D Salient Object Detection
with Depth Image Classification [2.578242050187029]
A novel RGB-D salient object detection model (SiaTrans) is proposed in this paper.
SiaTrans allows training on depth image quality classification at the same time as training on RGB-D saliency maps.
Experiments on nine RGB-D SOD benchmark datasets show that SiaTrans has the best overall performance and the least compared with recent state-of-the-art methods.
arXiv Detail & Related papers (2022-07-09T08:22:12Z) - Pyramidal Attention for Saliency Detection [30.554118525502115]
This paper exploits only RGB images, estimates depth from RGB, and leverages the intermediate depth features.
We employ a pyramidal attention structure to extract multi-level convolutional-transformer features to process initial stage representations.
We report significantly improved performance against 21 and 40 state-of-the-art SOD methods on eight RGB and RGB-D datasets.
arXiv Detail & Related papers (2022-04-14T06:57:46Z) - MobileSal: Extremely Efficient RGB-D Salient Object Detection [62.04876251927581]
This paper introduces a novel network, methodname, which focuses on efficient RGB-D salient object detection (SOD)
We propose an implicit depth restoration (IDR) technique to strengthen the feature representation capability of mobile networks for RGB-D SOD.
With IDR and CPR incorporated, methodnameperforms favorably against sArt methods on seven challenging RGB-D SOD datasets.
arXiv Detail & Related papers (2020-12-24T04:36:42Z) - Cascade Graph Neural Networks for RGB-D Salient Object Detection [41.57218490671026]
We study the problem of salient object detection (SOD) for RGB-D images using both color and depth information.
We introduce Cascade Graph Neural Networks(Cas-Gnn),a unified framework which is capable of comprehensively distilling and reasoning the mutual benefits between these two data sources.
Cas-Gnn achieves significantly better performance than all existing RGB-DSOD approaches on several widely-used benchmarks.
arXiv Detail & Related papers (2020-08-07T10:59:04Z) - Data-Level Recombination and Lightweight Fusion Scheme for RGB-D Salient
Object Detection [73.31632581915201]
We propose a novel data-level recombination strategy to fuse RGB with D (depth) before deep feature extraction.
A newly lightweight designed triple-stream network is applied over these novel formulated data to achieve an optimal channel-wise complementary fusion status between the RGB and D.
arXiv Detail & Related papers (2020-08-07T10:13:05Z) - Synergistic saliency and depth prediction for RGB-D saliency detection [76.27406945671379]
Existing RGB-D saliency datasets are small, which may lead to overfitting and limited generalization for diverse scenarios.
We propose a semi-supervised system for RGB-D saliency detection that can be trained on smaller RGB-D saliency datasets without saliency ground truth.
arXiv Detail & Related papers (2020-07-03T14:24:41Z) - Is Depth Really Necessary for Salient Object Detection? [50.10888549190576]
We make the first attempt in realizing an unified depth-aware framework with only RGB information as input for inference.
Not only surpasses the state-of-the-art performances on five public RGB SOD benchmarks, but also surpasses the RGBD-based methods on five benchmarks by a large margin.
arXiv Detail & Related papers (2020-05-30T13:40:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.