Modality-Agnostic Prompt Learning for Multi-Modal Camouflaged Object Detection

• URL: http://arxiv.org/abs/2604.12380v1
• Date: Tue, 14 Apr 2026 07:13:28 GMT
• Title: Modality-Agnostic Prompt Learning for Multi-Modal Camouflaged Object Detection
• Abstract summary: We propose a novel framework that generates modality-agnostic multi-modal prompts for the Segment Anything Model (SAM)<n>Specifically, we model multi-modal learning through interactions between a data-driven content domain and a knowledge-driven prompt domain.<n>We further introduce a lightweight Mask Refine Module to calibrate coarse predictions by incorporating fine-grained prompt cues.
• Score: 61.36976558603528
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Camouflaged Object Detection (COD) aims to segment objects that blend seamlessly into complex backgrounds, with growing interest in exploiting additional visual modalities to enhance robustness through complementary information. However, most existing approaches generally rely on modality-specific architectures or customized fusion strategies, which limit scalability and cross-modal generalization. To address this, we propose a novel framework that generates modality-agnostic multi-modal prompts for the Segment Anything Model (SAM), enabling parameter-efficient adaptation to arbitrary auxiliary modalities and significantly improving overall performance on COD tasks. Specifically, we model multi-modal learning through interactions between a data-driven content domain and a knowledge-driven prompt domain, distilling task-relevant cues into unified prompts for SAM decoding. We further introduce a lightweight Mask Refine Module to calibrate coarse predictions by incorporating fine-grained prompt cues, leading to more accurate camouflaged object boundaries. Extensive experiments on RGB-Depth, RGB-Thermal, and RGB-Polarization benchmarks validate the effectiveness and generalization of our modality-agnostic framework.

Related papers

• MODAL: Multi-Modal Object Re-ID via Model-Driven Sparse Decoupling and Text-Image Differential Filtering [51.99452481869329]
  MODAL is a novel multi-modal object re-identification framework grounded in coupled sparse coding theory and differential suppression principles.<n>A core component of MODAL is a Multi-modal Feature Sparse Decoupling module, developed in a model-driven deep unrolling manner.<n>Benefiting from the principled feature disentanglement, MODAL naturally mitigates performance degradation in incomplete-modality scenarios.<n>Experiments on four datasets demonstrate that MODAL achieves state-of-the-art performance with superior transparency.
  arXiv  Detail & Related papers  (2026-08-15T07:41:10Z)
• CMTFormer: Marrying Transformer with Hierarchical Information Interaction for RGB-Event Object Detection [17.487415888287]
  Event cameras capture sparse brightness changes with high temporal resolution and high dynamic range.<n>Previous multi-modal fusion techniques fail to handle the inherent heterogeneity between RGB frames and event streams.<n>We propose a Cross-Modal information inTeraction transFormer, which hierarchically integrates RGB and event information.
  arXiv  Detail & Related papers  (2026-06-28T00:49:05Z)
• Learning Progressive Adaptation for Multi-Modal Tracking [67.50696675353451]
  Multi-modal trackers are typically built by adopting pre-trained RGB models with parameter-efficient fine-tuning modules.<n>To address the issues, we propose to perform Progressive Adaptation for Multi-Modal Tracking (PATrack)<n>This innovative approach incorporates modality-dependent, modality-entangled, and task-level adapters, effectively bridging the gap in adapting RGB pre-trained networks to multi-modal data.
  arXiv  Detail & Related papers  (2026-03-22T07:25:54Z)
• Multi-Modal Building Change Detection for Large-Scale Small Changes: Benchmark and Baseline [16.08630411584202]
  Change detection in optical remote sensing imagery is susceptible to illumination fluctuations, seasonal changes, and variations in surface land-cover materials.<n>We introduce the Large-scale Small-change Multi-modal dataset (LSMD), a bi-temporal RGB-NIR building change detection benchmark dataset.<n>We further propose the Multi-modal Spectral Complementarity Network (MSCNet) to achieve effective cross-modal feature fusion.
  arXiv  Detail & Related papers  (2026-03-19T16:05:05Z)
• STMI: Segmentation-Guided Token Modulation with Cross-Modal Hypergraph Interaction for Multi-Modal Object Re-Identification [14.549172375231729]
  We propose STMI, a novel multi-modal learning framework consisting of three key components.<n>We demonstrate the effectiveness and robustness of our proposed STMI framework in multi-modal ReID scenarios.
  arXiv  Detail & Related papers  (2026-02-28T15:07:10Z)
• HyPSAM: Hybrid Prompt-driven Segment Anything Model for RGB-Thermal Salient Object Detection [75.406055413928]
  We propose a novel prompt-driven segment anything model (HyPSAM) for RGB-T SOD.<n> DFNet employs dynamic convolution and multi-branch decoding to facilitate adaptive cross-modality interaction.<n>Experiments on three public datasets demonstrate that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-09-23T07:32:11Z)
• AuxDet: Auxiliary Metadata Matters for Omni-Domain Infrared Small Target Detection [49.81255045696323]
  We present the Auxiliary Metadata Driven Infrared Small Target Detector (AuxDet)<n>AuxDet integrates metadata semantics with visual features, guiding adaptive representation learning for each sample.<n>Experiments on the challenging WideIRSTD-Full benchmark demonstrate that AuxDet consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-05-21T07:02:05Z)
• Adapting Segment Anything Model to Multi-modal Salient Object Detection with Semantic Feature Fusion Guidance [15.435695491233982]
  We propose a novel framework to explore and exploit the powerful feature representation and zero-shot generalization ability of the Segment Anything Model (SAM) for multi-modal salient object detection (SOD) We develop underlineSAM with seunderlinemantic funderlineeature fuunderlinesion guidancunderlinee (Sammese) In the image encoder, a multi-modal adapter is proposed to adapt the single-modal SAM to multi-modal information. Specifically, in the mask decoder, a semantic-geometric
  arXiv  Detail & Related papers  (2024-08-27T13:47:31Z)
• Exploiting Modality-Specific Features For Multi-Modal Manipulation Detection And Grounding [54.49214267905562]
  We construct a transformer-based framework for multi-modal manipulation detection and grounding tasks. Our framework simultaneously explores modality-specific features while preserving the capability for multi-modal alignment. We propose an implicit manipulation query (IMQ) that adaptively aggregates global contextual cues within each modality.
  arXiv  Detail & Related papers  (2023-09-22T06:55:41Z)
• RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentation [53.4319652364256]
  This paper presents the RefSAM model, which explores the potential of SAM for referring video object segmentation. Our proposed approach adapts the original SAM model to enhance cross-modality learning by employing a lightweight Cross-RValModal. We employ a parameter-efficient tuning strategy to align and fuse the language and vision features effectively.
  arXiv  Detail & Related papers  (2023-07-03T13:21:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.