SynCLIP: Synonym-Coherent Language-Image Pretraining for Robust Open-Vocabulary Dense Perception

• URL: http://arxiv.org/abs/2607.11008v1
• Date: Mon, 13 Jul 2026 02:16:17 GMT
• Title: SynCLIP: Synonym-Coherent Language-Image Pretraining for Robust Open-Vocabulary Dense Perception
• Abstract summary: SynCLIP is a Synonym-Coherent Language-Image Pretraining framework.<n>It enhances synonym-robust grounding for open-vocabulary dense perception.<n>It achieves state-of-the-art performance among CLIP-based OVDP methods.
• Score: 18.893901764545742
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open-vocabulary dense perception (OVDP) aims to localize objects unseen during training by leveraging textual knowledge. Despite the remarkable progress of recent CLIP-based approaches, we identify a critical limitation: synonym-induced grounding inconsistency, where semantically equivalent expressions yield disparate spatial attention patterns. This inconsistency undermines the robustness and performance of existing methods in real-world OVDP applications. To address this issue, we propose SynCLIP, a Synonym-Coherent Language-Image Pretraining framework that enhances synonym-robust grounding for OVDP. SynCLIP introduces a Semantic-consistent Spatial Attention alignment (SSA) module to enhance spatial attention consistency by minimizing discrepancies between attention maps of original and synonymous expressions. Furthermore, a Spatial Attention Refinement (SAR) module selectively strengthens the most semantically relevant spatial regions within aligned maps for more precise and stable grounding. To support synonym-coherent pretraining, we also construct a Synonym-Enriched Visual Corpus (SEViC), which augments each category with multiple synonyms and textual definitions. Extensive experiments on multiple benchmarks demonstrate that SynCLIP substantially improves grounding consistency under diverse linguistic variants and achieves state-of-the-art performance among CLIP-based OVDP methods. Code is available at https://github.com/Justlovesmile/SynCLIP.

Related papers

• Learning from Multimodal Pseudo-Labels for Robust Open-Vocabulary Instance and Panoptic Segmentation [11.82691757866647]
  This work addresses the challenge of open-vocabulary instance segmentation (OVIS) and open-set panoptic segmentation (OSPS)<n>Existing methods often suffer from noisy pseudo-masks, limited visual-textual grounding, and difficulty handling synonyms or out-of-vocabulary words.<n>We propose a multimodal framework that leverages pre-trained vision-language models for automatic pseudo-label generation, CLIP-guided synonym filtering, and GPT-based caption reconstruction.
  arXiv  Detail & Related papers  (2026-08-12T05:39:57Z)
• TraceCLIP: Recovering Local Semantics from Patch-to-CLS Contributions [20.430455131220544]
  We introduce TraceCLIP, a training-free framework that recovers latent patch-level semantic evidence.<n>We show that TraceCLIP achieves gains of 1.3 to 4.5 points in average mIoU over the strongest prior training-free methods.
  arXiv  Detail & Related papers  (2026-07-28T11:25:27Z)
• Structured-Condensed Prompt Tuning in Vision-Language Models for Fine-grained Image Recognition [38.294322671641545]
  Fine-grained image recognition is a significant challenge due to the substantial expertise and effort required for manual annotation.<n>Current methods treat class labels as isolated, discrete entities, overlooking the rich semantic relationships between them.<n>We propose Structured-Condensed Prompt Tuning ( SCPT), which enhances semantic structure modeling in prompt learning.
  arXiv  Detail & Related papers  (2026-07-07T12:09:49Z)
• Seeking Consensus: Geometric-Semantic On-the-Fly Recalibration for Open-Vocabulary Remote Sensing Semantic Segmentation [27.476148859056114]
  SeeCo is a plug-and-play framework to boost the performance of training-free OVSS models in remote sensing images.<n>It recalibrates arbitrary OVSS models on-the-fly by seeking dual consensus.<n>Experiments on eight remote sensing OVSS benchmarks show consistent gains.
  arXiv  Detail & Related papers  (2026-04-29T01:57:21Z)
• Unifying Heterogeneous Multi-Modal Remote Sensing Detection Via Language-Pivoted Pretraining [59.2578488860426]
  Heterogeneous multi-modal remote sensing object detection aims to accurately detect objects from diverse sensors.<n>Existing approaches largely adopt a late alignment paradigm, in which modality alignment and task-specific optimization are entangled during downstream fine-tuning.<n>We propose BabelRS, a unified language-pivoted pretraining framework that explicitly decouples modality alignment from downstream task learning.
  arXiv  Detail & Related papers  (2026-03-02T11:38:12Z)
• Context Patch Fusion With Class Token Enhancement for Weakly Supervised Semantic Segmentation [5.859471574310312]
  We propose the Context Patch Fusion with Class Token Enhancement (CPF-CTE) framework.<n>CPF-CTE exploits contextual relations among patches to enrich feature representations and improve segmentation.<n>We show that CPF-CTE consistently surpasses prior WSSS methods.
  arXiv  Detail & Related papers  (2026-01-21T07:12:23Z)
• Improving LLM Reasoning with Homophily-aware Structural and Semantic Text-Attributed Graph Compression [55.51959317490934]
  Large language models (LLMs) have demonstrated promising capabilities in Text-Attributed Graph (TAG) understanding.<n>We argue that graphs inherently contain rich structural and semantic information, and that their effective exploitation can unlock potential gains in LLMs reasoning performance.<n>We propose Homophily-aware Structural and Semantic Compression for LLMs (HS2C), a framework centered on exploiting graph homophily.
  arXiv  Detail & Related papers  (2026-01-13T03:35:18Z)
• Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception [71.26728044621458]
  DeCLIP is a novel framework that enhances CLIP by decoupling the self-attention module to obtain content'' and context'' features respectively.<n>It consistently achieves state-of-the-art performance across a broad spectrum of tasks, including 2D detection and segmentation, 3D instance segmentation, video instance segmentation, and 6D object pose estimation.
  arXiv  Detail & Related papers  (2025-08-15T06:43:51Z)
• Boosting Neural Language Inference via Cascaded Interactive Reasoning [38.125341836302525]
  Natural Language Inference (NLI) focuses on ascertaining the logical relationship between a given premise and hypothesis.<n>This task presents significant challenges due to inherent linguistic features such as diverse phrasing, semantic complexity, and contextual nuances.<n>We introduce the Cascaded Interactive Reasoning Network (CIRN), a novel architecture designed for deeper semantic comprehension in NLI.
  arXiv  Detail & Related papers  (2025-05-10T11:37:15Z)
• Spatial Semantic Recurrent Mining for Referring Image Segmentation [63.34997546393106]
  We propose Stextsuperscript2RM to achieve high-quality cross-modality fusion. It follows a working strategy of trilogy: distributing language feature, spatial semantic recurrent coparsing, and parsed-semantic balancing. Our proposed method performs favorably against other state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2024-05-15T00:17:48Z)
• Open-Vocabulary Segmentation with Semantic-Assisted Calibration [68.41025728960176]
  We study open-vocabulary segmentation (OVS) through calibrating in-vocabulary and domain-biased embedding space with contextual prior of CLIP.<n>We present a Semantic-assisted CAlibration Network (SCAN) to achieve state-of-the-art performance on open-vocabulary segmentation benchmarks.
  arXiv  Detail & Related papers  (2023-12-07T07:00:09Z)
• Contextualized Semantic Distance between Highly Overlapped Texts [85.1541170468617]
  Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation. This paper aims to address the issue with a mask-and-predict strategy. We take the words in the longest common sequence as neighboring words and use masked language modeling (MLM) to predict the distributions on their positions. Experiments on Semantic Textual Similarity show NDD to be more sensitive to various semantic differences, especially on highly overlapped paired texts.
  arXiv  Detail & Related papers  (2021-10-04T03:59:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.