DouC: Dual-Branch CLIP for Training-Free Open-Vocabulary Segmentation

• URL: http://arxiv.org/abs/2604.24997v1
• Date: Mon, 27 Apr 2026 20:59:01 GMT
• Title: DouC: Dual-Branch CLIP for Training-Free Open-Vocabulary Segmentation
• Abstract summary: DouC is a training-free dual-branch CLIP framework that decomposes dense prediction into two complementary components.<n> OG-CLIP improves patch-level reliability via lightweight, inference-time token gating.<n>FADE-CLIP injects external structural priors through proxy attention guided by frozen vision foundation models.
• Score: 0.6015898117103069
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Open-vocabulary semantic segmentation requires assigning pixel-level semantic labels while supporting an open and unrestricted set of categories. Training-free CLIP-based approaches preserve strong zero-shot generalization but typically rely on a single inference mechanism, limiting their ability to jointly address unreliable local tokens and insufficient spatial coherence. We propose DouC, a training-free dual-branch CLIP framework that decomposes dense prediction into two complementary components. OG-CLIP improves patch-level reliability via lightweight, inference-time token gating, while FADE-CLIP injects external structural priors through proxy attention guided by frozen vision foundation models. The two branches are fused at the logit level, enabling local token reliability and structure-aware patch interactions to jointly influence final predictions, with optional instance-aware correction applied as post-processing. DouC introduces no additional learnable parameters, requires no retraining, and preserves CLIP's zero-shot generalization. Extensive experiments across eight benchmarks and multiple CLIP backbones demonstrate that DouC consistently outperforms prior training-free methods and scales favorably with model capacity.

Related papers

• SCI-CLIP: Segment-Centric Inference with Reference Memory for Training-Free Open-Vocabulary Segmentation [0.6015898117103069]
  We present SCI-CLIP, a segment-centric inference framework built around the principle that the same region abstraction should organize all stages of dense open-vocabulary prediction.<n> SCI-CLIP turns frozen CLIP-style features into spatially coherent, context-aware, and retrieval-compatible dense predictions without any training.
  arXiv  Detail & Related papers  (2026-08-06T05:47:30Z)
• TraceCLIP: Recovering Local Semantics from Patch-to-CLS Contributions [20.430455131220544]
  We introduce TraceCLIP, a training-free framework that recovers latent patch-level semantic evidence.<n>We show that TraceCLIP achieves gains of 1.3 to 4.5 points in average mIoU over the strongest prior training-free methods.
  arXiv  Detail & Related papers  (2026-07-28T11:25:27Z)
• Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration [72.0672328514289]
  Multimodal learning often grapples with the challenge of low-quality data.<n>We propose a unified framework, termed Conformal Predictive Self-Calibration.<n>Our framework consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-05T14:48:52Z)
• Decouple and Rectify: Semantics-Preserving Structural Enhancement for Open-Vocabulary Remote Sensing Segmentation [23.298715255853782]
  Open-vocabulary semantic segmentation in the remote sensing (RS) field requires both language-aligned recognition and fine-grained spatial delineation.<n>Recent methods attempt to compensate for this by introducing RS-pretrained DINO features.<n>We propose DR-Seg, a novel de-couple-and-rectify framework.
  arXiv  Detail & Related papers  (2026-04-02T13:15:05Z)
• BOFA: Bridge-Layer Orthogonal Low-Rank Fusion for CLIP-Based Class-Incremental Learning [84.56022893225422]
  Class-Incremental Learning (CIL) aims to continually learn new categories without forgetting previously acquired knowledge.<n>Applying vision-language models such as CLIP to CIL poses two major challenges: (1) adapting to downstream tasks often requires additional learnable modules, increasing model complexity and susceptibility to forgetting; and (2) while multi-modal representations offer complementary strengths, existing methods have yet to fully realize their potential in effectively integrating visual and textual modalities.
  arXiv  Detail & Related papers  (2025-11-14T15:51:40Z)
• Hierarchical Self-Supervised Representation Learning for Depression Detection from Speech [51.14752758616364]
  Speech-based depression detection (SDD) is a promising, non-invasive alternative to traditional clinical assessments.<n>We propose HAREN-CTC, a novel architecture that integrates multi-layer SSL features using cross-attention within a multitask learning framework.<n>The model achieves state-of-the-art macro F1-scores of 0.81 on DAIC-WOZ and 0.82 on MODMA, outperforming prior methods across both evaluation scenarios.
  arXiv  Detail & Related papers  (2025-10-05T09:32:12Z)
• Continual Learning on CLIP via Incremental Prompt Tuning with Intrinsic Textual Anchors [50.7383184560431]
  Continual learning (CL) enables deep networks to acquire new knowledge while avoiding catastrophic forgetting.<n>We propose a concise CL approach for CLIP based on incremental prompt tuning.<n>We show that our bidirectional supervision strategy enables more effective learning of new knowledge while reducing forgetting.
  arXiv  Detail & Related papers  (2025-05-27T03:51:37Z)
• Self-Calibrated CLIP for Training-Free Open-Vocabulary Segmentation [19.749490092520006]
  Self-Calibrated CLIP (SC-CLIP) is a training-free method that calibrates CLIP to produce finer representations.<n>SC-CLIP boosts the performance of vanilla CLIP ViT-L/14 by 6.8 times.
  arXiv  Detail & Related papers  (2024-11-24T15:14:05Z)
• Explore the Potential of CLIP for Training-Free Open Vocabulary Semantic Segmentation [38.16802763051431]
  We propose CLIPtrase, a training-free semantic segmentation strategy. It enhances local feature awareness through recalibrated self-correlation among patches. Experiments show that we are 22.3% ahead of CLIP on average on 9 segmentation benchmarks.
  arXiv  Detail & Related papers  (2024-07-11T08:12:16Z)
• CLAP4CLIP: Continual Learning with Probabilistic Finetuning for Vision-Language Models [23.398619576886375]
  Continual learning (CL) aims to help deep neural networks learn new knowledge while retaining what has been learned. Our work proposes Continual LeArning with Probabilistic finetuning (CLAP) - a probabilistic modeling framework over visual-guided text features per task.
  arXiv  Detail & Related papers  (2024-03-28T04:15:58Z)
• CLIPood: Generalizing CLIP to Out-of-Distributions [73.86353105017076]
  Contrastive language-image pre-training (CLIP) models have shown impressive zero-shot ability, but the further adaptation of CLIP on downstream tasks undesirably degrades OOD performances. We propose CLIPood, a fine-tuning method that can adapt CLIP models to OOD situations where both domain shifts and open classes may occur on unseen test data. Experiments on diverse datasets with different OOD scenarios show that CLIPood consistently outperforms existing generalization techniques.
  arXiv  Detail & Related papers  (2023-02-02T04:27:54Z)
• DenseCLIP: Extract Free Dense Labels from CLIP [130.3830819077699]
  Contrastive Language-Image Pre-training (CLIP) has made a remarkable breakthrough in open-vocabulary zero-shot image recognition. DenseCLIP+ surpasses SOTA transductive zero-shot semantic segmentation methods by large margins. Our finding suggests that DenseCLIP can serve as a new reliable source of supervision for dense prediction tasks.
  arXiv  Detail & Related papers  (2021-12-02T09:23:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.