Phantom-Insight: Adaptive Multi-cue Fusion for Video Camouflaged Object Detection with Multimodal LLM

• URL: http://arxiv.org/abs/2509.06422v1
• Date: Mon, 08 Sep 2025 08:17:47 GMT
• Title: Phantom-Insight: Adaptive Multi-cue Fusion for Video Camouflaged Object Detection with Multimodal LLM
• Authors: Hua Zhang, Changjiang Luo, Ruoyu Chen,
• Abstract summary: Video camouflaged object detection (VCOD) is challenging due to dynamic environments.<n>Existing methods face two main issues: (1) SAM-based methods struggle to separate camouflaged object edges due to model freezing, and (2) MLLM-based methods suffer from poor object separability as large language models merge foreground and background.<n>We propose a novel VCOD method based on SAM and MLLM, called Phantom-Insight.
• Score: 5.817528321487845
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video camouflaged object detection (VCOD) is challenging due to dynamic environments. Existing methods face two main issues: (1) SAM-based methods struggle to separate camouflaged object edges due to model freezing, and (2) MLLM-based methods suffer from poor object separability as large language models merge foreground and background. To address these issues, we propose a novel VCOD method based on SAM and MLLM, called Phantom-Insight. To enhance the separability of object edge details, we represent video sequences with temporal and spatial clues and perform feature fusion via LLM to increase information density. Next, multiple cues are generated through the dynamic foreground visual token scoring module and the prompt network to adaptively guide and fine-tune the SAM model, enabling it to adapt to subtle textures. To enhance the separability of objects and background, we propose a decoupled foreground-background learning strategy. By generating foreground and background cues separately and performing decoupled training, the visual token can effectively integrate foreground and background information independently, enabling SAM to more accurately segment camouflaged objects in the video. Experiments on the MoCA-Mask dataset show that Phantom-Insight achieves state-of-the-art performance across various metrics. Additionally, its ability to detect unseen camouflaged objects on the CAD2016 dataset highlights its strong generalization ability.

Related papers

• Evaluating SAM2 for Video Semantic Segmentation [60.157605818225186]
  The Anything Model 2 (SAM2) has proven to be a powerful foundation model for promptable visual object segmentation in both images and videos.<n>This paper explores the extension of SAM2 to dense Video Semantic (VSS)<n>Our experiments suggest that leveraging SAM2 enhances overall performance in VSS, primarily due to its precise predictions of object boundaries.
  arXiv  Detail & Related papers  (2025-12-01T15:15:16Z)
• Object-Aware Video Matting with Cross-Frame Guidance [35.785998735049006]
  We present a trimap-free Object-Aware Video Matting (OAVM) framework, which can perceive different objects, enabling joint recognition of foreground objects and refinement of edge details.<n>Specifically, we propose an Object-Guided Correction and Refinement (OGCR) module, which employs cross-frame guidance to aggregate object-level instance information into pixel-level detail features.<n>We also design a Sequential Foreground Merging augmentation strategy to diversify sequential scenarios and enhance capacity of the network for object discrimination.
  arXiv  Detail & Related papers  (2025-03-03T07:40:32Z)
• TWIST & SCOUT: Grounding Multimodal LLM-Experts by Forget-Free Tuning [54.033346088090674]
  We introduce TWIST & SCOUT, a framework that equips pre-trained MLLMs with visual grounding ability.<n>To fine-tune the model effectively, we generate a high-quality synthetic dataset we call SCOUT.<n>This dataset provides rich supervision signals, describing a step-by-step multimodal reasoning process.
  arXiv  Detail & Related papers  (2024-10-14T13:35:47Z)
• Hierarchical Graph Interaction Transformer with Dynamic Token Clustering for Camouflaged Object Detection [57.883265488038134]
  We propose a hierarchical graph interaction network termed HGINet for camouflaged object detection. The network is capable of discovering imperceptible objects via effective graph interaction among the hierarchical tokenized features. Our experiments demonstrate the superior performance of HGINet compared to existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-08-27T12:53:25Z)
• Camouflaged Object Detection with Feature Grafting and Distractor Aware [9.791590363932519]
  We propose a novel Feature Grafting and Distractor Aware network (FDNet) to handle the Camouflaged Object Detection task. Specifically, we use CNN and Transformer to encode multi-scale images in parallel. A Distractor Aware Module is designed to explicitly model the two possible distractors in the COD task to refine the coarse camouflage map.
  arXiv  Detail & Related papers  (2023-07-08T09:37:08Z)
• RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentation [53.4319652364256]
  This paper presents the RefSAM model, which explores the potential of SAM for referring video object segmentation. Our proposed approach adapts the original SAM model to enhance cross-modality learning by employing a lightweight Cross-RValModal. We employ a parameter-efficient tuning strategy to align and fuse the language and vision features effectively.
  arXiv  Detail & Related papers  (2023-07-03T13:21:58Z)
• Dense Video Object Captioning from Disjoint Supervision [77.47084982558101]
  We propose a new task and model for dense video object captioning. This task unifies spatial and temporal localization in video. We show how our model improves upon a number of strong baselines for this new task.
  arXiv  Detail & Related papers  (2023-06-20T17:57:23Z)
• Feature Aggregation and Propagation Network for Camouflaged Object Detection [42.33180748293329]
  Camouflaged object detection (COD) aims to detect/segment camouflaged objects embedded in the environment. Several COD methods have been developed, but they still suffer from unsatisfactory performance due to intrinsic similarities between foreground objects and background surroundings. We propose a novel Feature Aggregation and propagation Network (FAP-Net) for camouflaged object detection.
  arXiv  Detail & Related papers  (2022-12-02T05:54:28Z)
• MFFN: Multi-view Feature Fusion Network for Camouflaged Object Detection [10.04773536815808]
  We propose a behavior-inspired framework, called Multi-view Feature Fusion Network (MFFN), which mimics the human behaviors of finding indistinct objects in images. MFFN captures critical edge and semantic information by comparing and fusing extracted multi-view features. Our method performs favorably against existing state-of-the-art methods via training with the same data.
  arXiv  Detail & Related papers  (2022-10-12T16:12:58Z)
• Segmenting Moving Objects via an Object-Centric Layered Representation [100.26138772664811]
  We introduce an object-centric segmentation model with a depth-ordered layer representation. We introduce a scalable pipeline for generating synthetic training data with multiple objects. We evaluate the model on standard video segmentation benchmarks.
  arXiv  Detail & Related papers  (2022-07-05T17:59:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.