MARS-CLIP: Multi-Resolution and Attention Refined Zero-Shot Image Segmentation

• URL: http://arxiv.org/abs/2609.08283v1
• Date: Tue, 08 Sep 2026 05:53:17 GMT
• Title: MARS-CLIP: Multi-Resolution and Attention Refined Zero-Shot Image Segmentation
• Abstract summary: Contrastive Language-Image Pre-training (CLIP) has demonstrated impressive capabilities in zero-shot transfer but often struggles with dense prediction tasks due to low resolution and loss of structural information.<n>We propose MARS-CLIP, a novel framework for zero-shot semantic segmentation.<n>Our approach introduces two key strategies: (i) a multi-resolution feature extraction module that fuses local fine-grained features with global context to overcome input resolution constraints, and (ii) an attention refinement mechanism that injects spatial and color biases from intermediate layers into the final self-attention block to accurately restore object boundaries.
• Score: 1.3999481573773072
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Contrastive Language-Image Pre-training (CLIP) has demonstrated impressive capabilities in zero-shot transfer but often struggles with dense prediction tasks due to low spatial resolution and the loss of structural information. To address these limitations, we propose MARS-CLIP (Multi-resolution and Attention Refined Segmentation for CLIP), a novel framework for zero-shot semantic segmentation. Our approach introduces two key strategies: (i) a multi-resolution feature extraction module that fuses local fine-grained features with global context to overcome input resolution constraints, and (ii) an attention refinement mechanism that injects spatial and color biases from intermediate layers into the final self-attention block to accurately restore object boundaries. A set of experiments on six public datasets demonstrates that MARS-CLIP significantly outperforms state-of-the-art methods.

Related papers

• Learning to Focus and Precise Cropping: A Reinforcement Learning Framework with Information Gaps and Grounding Loss for MLLMs [58.32326205291745]
  MLLMs autonomously utilize image cropping tool to analyze regions of interest for question answering.<n>We demonstrate the model's strong reliance on global input and its weak dependence on the details within the cropped region.<n>We propose a novel two-stage reinforcement learning framework that does not require supervision.
  arXiv  Detail & Related papers  (2026-03-29T03:18:57Z)
• SSR: Semantic and Spatial Rectification for CLIP-based Weakly Supervised Segmentation [19.962252347191523]
  This paper proposes a novel Semantic and Spatial Rectification (SSR) method to address the limitations of existing CLIP-based weakly supervised semantic segmentation approaches.<n>Specifically, at the semantic level, the Cross-Modal Prototype Alignment (CMPA) establishes a contrastive learning mechanism to enforce feature space alignment.<n>At the spatial level, the Superpixel-Guided Correction (SGC) leverages superpixel-based spatial priors to precisely filter out interference from non-target regions during affinity propagation.
  arXiv  Detail & Related papers  (2025-12-01T14:06:50Z)
• Target Refocusing via Attention Redistribution for Open-Vocabulary Semantic Segmentation: An Explainability Perspective [47.99651635870674]
  We propose a training-free approach that emulates human distraction-refocusing behavior to redirect attention from distraction tokens back to target regions.<n>Our method achieves SOTA performance on eight benchmarks while maintaining high inference efficiency.
  arXiv  Detail & Related papers  (2025-11-20T09:16:33Z)
• MPCM-Net: Multi-scale network integrates partial attention convolution with Mamba for ground-based cloud image segmentation [13.137436418148896]
  Ground-based cloud image segmentation is a critical research domain for photovoltaic power forecasting.<n>We propose MPCM-Net, a Multi-scale network that integrates Partial attention Convolutions with Mamba architectures to enhance segmentation accuracy and computational efficiency.<n>As a key contribution to the community, we also introduce and release a dataset CSRC, which is a clear-label, fine-grained segmentation benchmark designed to overcome the critical limitations of existing public datasets.
  arXiv  Detail & Related papers  (2025-11-12T06:17:49Z)
• SRSR: Enhancing Semantic Accuracy in Real-World Image Super-Resolution with Spatially Re-Focused Text-Conditioning [59.013863248600046]
  We propose a spatially re-focused super-resolution framework that refines text conditioning at inference time.<n>Second, we introduce a Spatially Targeted-Free Guidance mechanism that selectively bypasses text influences on ungrounded pixels to prevent hallucinations.
  arXiv  Detail & Related papers  (2025-10-26T05:03:55Z)
• FineRS: Fine-grained Reasoning and Segmentation of Small Objects with Reinforcement Learning [62.11389260206383]
  textscFineRS is a two-stage MLLM-based reinforcement learning framework for segmenting extremely small objects.<n>We present textscFineRS-4k, a new dataset for evaluating MLLMs on attribute-level reasoning and pixel-level segmentation on subtle, small-scale targets.
  arXiv  Detail & Related papers  (2025-10-24T10:14:17Z)
• ResCLIP: Residual Attention for Training-free Dense Vision-language Inference [27.551367463011008]
  Cross-correlation of self-attention in CLIP's non-final layers also exhibits localization properties. We propose the Residual Cross-correlation Self-attention (RCS) module, which leverages the cross-correlation self-attention from intermediate layers to remold the attention in the final block. The RCS module effectively reorganizes spatial information, unleashing the localization potential within CLIP for dense vision-language inference.
  arXiv  Detail & Related papers  (2024-11-24T14:14:14Z)
• Point Cloud Understanding via Attention-Driven Contrastive Learning [64.65145700121442]
  Transformer-based models have advanced point cloud understanding by leveraging self-attention mechanisms. PointACL is an attention-driven contrastive learning framework designed to address these limitations. Our method employs an attention-driven dynamic masking strategy that guides the model to focus on under-attended regions.
  arXiv  Detail & Related papers  (2024-11-22T05:41:00Z)
• ClearCLIP: Decomposing CLIP Representations for Dense Vision-Language Inference [32.852004564832455]
  We re-investigate the architecture of CLIP, and identify residual connections as the primary source of noise that degrades segmentation quality. We propose ClearCLIP, a novel approach that decomposes CLIP's representations to enhance open-vocabulary semantic segmentation.
  arXiv  Detail & Related papers  (2024-07-17T09:52:20Z)
• Spatial Structure Constraints for Weakly Supervised Semantic Segmentation [100.0316479167605]
  A class activation map (CAM) can only locate the most discriminative part of objects. We propose spatial structure constraints (SSC) for weakly supervised semantic segmentation to alleviate the unwanted object over-activation of attention expansion. Our approach achieves 72.7% and 47.0% mIoU on the PASCAL VOC 2012 and COCO datasets, respectively.
  arXiv  Detail & Related papers  (2024-01-20T05:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.