LAGO: Language-Guided Adaptive Object-Region Focus for Zero-Shot Visual-Text Alignment

• URL: http://arxiv.org/abs/2605.08156v1
• Date: Mon, 04 May 2026 09:07:00 GMT
• Title: LAGO: Language-Guided Adaptive Object-Region Focus for Zero-Shot Visual-Text Alignment
• Abstract summary: In fine-grained settings, the relevant evidence often lies in localized parts, attributes, or textures rather than in the full image.<n>Recent localized visual-text alignment methods address this by comparing class descriptions with multiple image regions.<n>We propose LAGO, a framework for efficient and robust zero-shot localized visual-text alignment.
• Score: 11.785675631199455
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Zero-shot recognition aims to classify an image by selecting the most compatible label description from a set of candidate classes without any task-specific supervision. In fine-grained settings, however, the relevant evidence often lies in localized parts, attributes, or textures rather than in the full image, making whole-image alignment suboptimal. Recent localized visual-text alignment methods address this by comparing class descriptions with multiple image regions, but they typically rely on large sets of random or redundant crops, increasing inference cost and introducing many highly redundant or weakly relevant candidates. Moreover, introducing semantic guidance too early can create an error-amplifying feedback process in which inaccurate intermediate predictions bias later localization and reinforce subsequent mistakes; we refer to this failure mode as the prediction loop. We propose LAGO (LAnguage-Guided adaptive Object-region focus), a framework for efficient and robust zero-shot localized visual-text alignment. LAGO first performs class-agnostic object-centric candidate discovery to obtain a stable visual initialization, and then applies adaptive language-guided refinement with the strength of semantic guidance controlled by intermediate confidence. It further combines object-level, contextual, and full-image evidence through an effective object-context dual-channel aggregation strategy. Extensive experiments show that LAGO consistently achieves state-of-the-art performance on standard zero-shot benchmarks and challenging distribution-shift settings, while requiring substantially fewer candidate regions at inference time.

Related papers

• ExACT: Exemplar-Driven Calibrated Refinement for Training-Free Visual Grounding in Remote Sensing Images [51.53960096934913]
  Remote sensing visual grounding (RSVG) aims to locate specific objects in high-resolution RS imagery using free-form natural language descriptions.<n>Recent advances in multimodal large language models (MLLMs) show great potential for such open-vocabulary RSVG.<n>We propose Exemplar-driven Calibrated Refinement (ExACT), a training-free framework driven by a one-shot visual prompting mechanism.
  arXiv  Detail & Related papers  (2026-06-27T13:50:39Z)
• FOCUS: Forcing In-Context Object Localization through Visual Support Constraints and Policy Optimization [9.2112812802555]
  In-context localization (ICL) seeks to localize a target object specified by a small set of support examples in a query image.<n>Existing methods are fragile and rely on explicit category supervision.<n>We introduce a two-stage training framework that explicitly optimize in-context attention between support bounding boxes and query images.
  arXiv  Detail & Related papers  (2026-05-29T10:53:52Z)
• Leveraging Hierarchical Image-Text Misalignment for Universal Fake Image Detection [58.927873049646024]
  We show that fake images cannot be properly aligned with corresponding captions compared to real images.<n>We propose a simple yet effective ITEM by leveraging the image-text misalignment in a joint visual-language space as discriminative clues.
  arXiv  Detail & Related papers  (2025-11-01T06:51:14Z)
• Towards Fine-Grained Vision-Language Alignment for Few-Shot Anomaly Detection [65.29550320117526]
  We propose a novel framework named FineGrainedAD to improve anomaly localization performance.<n> Experiments demonstrate that the proposed FineGrainedAD achieves superior overall performance in few-shot settings.
  arXiv  Detail & Related papers  (2025-10-30T13:09:00Z)
• Enhancing CLIP Robustness via Cross-Modality Alignment [54.01929554563447]
  We propose Cross-modality Alignment, an optimal transport-based framework for vision-language models.<n> COLA restores global image-text alignment and local structural consistency in the feature space.<n> COLA is training-free and compatible with existing fine-tuned models.
  arXiv  Detail & Related papers  (2025-10-28T03:47:44Z)
• Constrained Prompt Enhancement for Improving Zero-Shot Generalization of Vision-Language Models [57.357091028792325]
  Vision-language models (VLMs) pre-trained on web-scale data exhibit promising zero-shot generalization but often suffer from semantic misalignment.<n>We propose a novel constrained prompt enhancement (CPE) method to improve visual-textual alignment.<n>Our approach consists of two key components: Topology-Guided Synonymous Semantic Generation (TGSSG) and Category-Agnostic Discriminative Region Selection (CADRS)
  arXiv  Detail & Related papers  (2025-08-24T15:45:22Z)
• Weakly-Supervised Image Forgery Localization via Vision-Language Collaborative Reasoning Framework [16.961220047066792]
  ViLaCo is a vision-language collaborative reasoning framework that introduces auxiliary semantic supervision distilled from pre-trained vision-language models.<n>ViLaCo substantially outperforms existing WSIFL methods, achieving state-of-the-art performance in both detection and localization accuracy.
  arXiv  Detail & Related papers  (2025-08-02T12:14:29Z)
• IteRPrimE: Zero-shot Referring Image Segmentation with Iterative Grad-CAM Refinement and Primary Word Emphasis [46.502962768034166]
  Zero-shot Referring Image identifies the instance mask that best aligns with a referring expression without training and fine-tuning.<n>Previous CLIP-based models exhibit a notable reduction in their capacity to discern relative spatial relationships of objects.<n>IteRPrimE outperforms previous state-of-the-art zero-shot methods, particularly excelling in out-of-domain scenarios.
  arXiv  Detail & Related papers  (2025-03-02T15:19:37Z)
• Do LLMs Understand Visual Anomalies? Uncovering LLM's Capabilities in Zero-shot Anomaly Detection [18.414762007525137]
  Large vision-language models (LVLMs) are proficient in deriving visual representations guided by natural language.<n>Recent explorations have utilized LVLMs to tackle zero-shot visual anomaly detection (VAD) challenges.<n>We present ALFA, a training-free approach designed to address these challenges via a unified model.
  arXiv  Detail & Related papers  (2024-04-15T10:42:22Z)
• Bootstrap Fine-Grained Vision-Language Alignment for Unified Zero-Shot Anomaly Localization [63.61093388441298]
  Contrastive Language-Image Pre-training models have shown promising performance on zero-shot visual recognition tasks. In this work, we propose AnoCLIP for zero-shot anomaly localization.
  arXiv  Detail & Related papers  (2023-08-30T10:35:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.