Weakly-Supervised RGB-D Salient Object Detection via SAM-driven Pseudo Annotation and State Space Interaction-based Diffusion

• URL: http://arxiv.org/abs/2607.15041v1
• Date: Thu, 16 Jul 2026 14:21:12 GMT
• Title: Weakly-Supervised RGB-D Salient Object Detection via SAM-driven Pseudo Annotation and State Space Interaction-based Diffusion
• Abstract summary: We propose a novel scribble-supervised RGB-D SOD method, consisting of a Segment Anything Model (SAM)-driven pseudo annotation generation method (emphSAM-PAG) and a state space interaction-based conditional diffusion model (emph$S2$Diff)<n>Our method outperforms relevant scribble-supervised methods and achieves competitive performance compared to fully-supervised methods on seven datasets.
• Score: 48.141883885276634
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Weakly-supervised RGB-D Salient Object Detection (SOD) is explored to reduce the heavy burden of pixel-level annotations. But scribble annotations lack the structure and details of objects, resulting in inaccurate saliency maps. In this paper, we propose a novel scribble-supervised RGB-D SOD method, consisting of a Segment Anything Model (SAM)-driven pseudo annotation generation method (\emph{SAM-PAG}) and a state space interaction-based conditional diffusion model (\emph{$S^2$Diff}). Specifically, SAM-PAG is tailored to address the issue of sparse supervision information. In SAM-PAG, we adopt the advanced SAM to expand sparse scribbles to dense pixel-level pseudo annotations through the dual-branch structure and the consistency of segmentation masks. In $S^2$Diff, we adopt the diffusion model to iteratively refine the noisy saliency maps with the guidance of conditional information, generating accurate saliency maps. Naturally, the core of our $S^2$Diff lies in the acquisition of conditional features and the denoising of saliency maps. For the former, we employ a cross-modal conditional generation module to interweave cross-modal features through frequency integration and implicit-explicit state space interaction, effectively achieving global conditional features. For the latter, we employ a context injection module to mitigate noise interference and to enhance object information with the conditional context. With the close cooperation of SAM-PAG and $S^2$Diff, our method outperforms relevant scribble-supervised methods and achieves competitive performance compared to fully-supervised methods on seven datasets. The code and results of our method are available at https://github.com/Switch457/WeakS2Diff_SOD.

Related papers

• SPDA-SAM: A Self-prompted Depth-Aware Segment Anything Model for Instance Segmentation [12.878470455789945]
  We propose a Self-prompted Depth-Aware SAM (SPDA-SAM) for instance segmentation.<n> Specifically, we design a Semantic-Spatial Self-prompt Module (SSSPM) which extracts the semantic and spatial prompts from the image encoder and the mask decoder of SAM.<n>We also introduce a Coarse-to-Fine RGB-D Fusion Module (C2FFM) in which the features extracted from a monocular RGB image and the depth map estimated from it are fused.
  arXiv  Detail & Related papers  (2026-02-06T03:01:41Z)
• CADTrack: Learning Contextual Aggregation with Deformable Alignment for Robust RGBT Tracking [68.71826342377004]
  RGB-Thermal (RGBT) tracking aims to exploit visible and thermal infrared modalities for robust all-weather object tracking.<n>Existing RGBT trackers struggle to resolve modality discrepancies, which poses great challenges for robust feature representation.<n>We propose a novel Contextual Aggregation with Deformable Alignment framework called CADTrack for RGBT Tracking.
  arXiv  Detail & Related papers  (2025-11-22T08:10:02Z)
• HyPSAM: Hybrid Prompt-driven Segment Anything Model for RGB-Thermal Salient Object Detection [75.406055413928]
  We propose a novel prompt-driven segment anything model (HyPSAM) for RGB-T SOD.<n> DFNet employs dynamic convolution and multi-branch decoding to facilitate adaptive cross-modality interaction.<n>Experiments on three public datasets demonstrate that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-09-23T07:32:11Z)
• DRRNet: Macro-Micro Feature Fusion and Dual Reverse Refinement for Camouflaged Object Detection [3.4710674889857787]
  DRRNet is a four-stage architecture characterized by a "context-fusion-refinement" pipeline to address these issues.<n>We introduce an Omni-Context Feature Extraction Module to capture global camouflage patterns.<n>We then design a module for forming dual representations of scene understanding and structural awareness.
  arXiv  Detail & Related papers  (2025-05-14T06:03:53Z)
• PolSAM: Polarimetric Scattering Mechanism Informed Segment Anything Model [83.35198885088093]
  PolSAR data presents unique challenges due to its rich and complex characteristics.<n>Existing data representations, such as complex-valued data, polarimetric features, and amplitude images, are widely used.<n>Most feature extraction networks for PolSAR are small, limiting their ability to capture features effectively.<n>We propose the Polarimetric Scattering Mechanism-Informed SAM (PolSAM), an enhanced Segment Anything Model (SAM) that integrates domain-specific scattering characteristics and a novel prompt generation strategy.
  arXiv  Detail & Related papers  (2024-12-17T09:59:53Z)
• PointSAM: Pointly-Supervised Segment Anything Model for Remote Sensing Images [16.662173255725463]
  We propose a novel Pointly-supervised Segment Anything Model named PointSAM.<n>We conduct experiments on RSI datasets, including WHU, HRSID, and NWPU VHR-10.<n>The results show that our method significantly outperforms direct testing with SAM, SAM2, and other comparison methods.
  arXiv  Detail & Related papers  (2024-09-20T11:02:18Z)
• Learning Spatial-Semantic Features for Robust Video Object Segmentation [108.045326229865]
  We propose a robust video object segmentation framework that learns spatial-semantic features and discriminative object queries.<n>The proposed method achieves state-of-the-art performance on benchmark data sets, including the DAVIS 2017 test (textbf87.8%), YoutubeVOS 2019 (textbf88.1%), MOSE val (textbf74.0%), and LVOS test (textbf73.0%)
  arXiv  Detail & Related papers  (2024-07-10T15:36:00Z)
• The devil is in the object boundary: towards annotation-free instance segmentation using Foundation Models [24.53385855664792]
  In object detection and instance segmentation, foundation models such as SAM and DINO struggle to achieve satisfactory performance. We propose $textbfZip$ which $textbfZ$ips up CL$textbfip$ and SAM in a novel classification-first-then-discovery pipeline. Our Zip significantly boosts SAM's mask AP on COCO dataset by 12.5% and establishes state-of-the-art performance in various settings.
  arXiv  Detail & Related papers  (2024-04-18T07:22:38Z)
• MUNet: Motion Uncertainty-aware Semi-supervised Video Object Segmentation [31.100954335785026]
  We advocate the return of the emphmotion information and propose a motion uncertainty-aware framework (MUNet) for semi-supervised video object segmentation. We introduce a motion-aware spatial attention module to effectively fuse the motion feature with the semantic feature. We achieve $76.5%$ $mathcalJ & mathcalF$ only using DAVIS17 for training, which significantly outperforms the textitSOTA methods under the low-data protocol.
  arXiv  Detail & Related papers  (2021-11-29T16:01:28Z)
• Layout-to-Image Translation with Double Pooling Generative Adversarial Networks [76.83075646527521]
  We propose a novel Double Pooing GAN (DPGAN) for generating photo-realistic and semantically-consistent results from the input layout. We also propose a novel Double Pooling Module (DPM), which consists of the Square-shape Pooling Module (SPM) and the Rectangle-shape Pooling Module ( RPM)
  arXiv  Detail & Related papers  (2021-08-29T19:55:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.