FAST-GOAL: Fast and Efficient Global-local Object Alignment Learning

• URL: http://arxiv.org/abs/2605.26615v1
• Date: Tue, 26 May 2026 06:52:46 GMT
• Title: FAST-GOAL: Fast and Efficient Global-local Object Alignment Learning
• Abstract summary: We present FAST-GOAL, an efficient fine-tuning method that enhances ability of CLIP to handle lengthy text.<n>Our method consists of two key components. First, Fast Local Image-Sentence Matching (FLISM) efficiently extracts local image regions.<n>Second, Token Similarity-based Learning (TSL) maximizes the similarity between patch tokens from specific regions in the image and their corresponding region embeddings.
• Score: 11.395181681423892
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models such as CLIP have shown impressive capabilities in aligning images and text, but they often struggle with lengthy and detailed text descriptions due to pre-training on short and concise captions. We present FAST-GOAL (Fast and Efficient Global-local Object Alignment Learning), an efficient fine-tuning method that enhances ability of CLIP to handle lengthy text through global-local semantic alignment. Our method consists of two key components. First, Fast Local Image-Sentence Matching (FLISM) efficiently extracts local image regions through object detection and spatial division, then matches them with corresponding sentences. Second, Token Similarity-based Learning (TSL) maximizes the similarity between patch tokens from specific regions in the image and their corresponding region embeddings, applying the same principle to text, which enhances the ability of the model to capture detailed correspondences. Additionally, we introduce GLIT100k, a dataset that provides both global image-lengthy caption pairs and context-derived local pairs, where local descriptions are extracted from global captions to maintain semantic coherence. Through extensive experiments on long caption datasets (DOCCI, DCI) and short caption datasets (MSCOCO, Flickr30k), we demonstrate that FAST-GOAL achieves significant improvements over baselines, enabling effective adaptation of CLIP to detailed textual descriptions while maintaining computational efficiency.

Related papers

• MulCLIP: A Multi-level Alignment Framework for Enhancing Fine-grained Long-context CLIP [4.6096940605642915]
  MulCLIP is an end-to-end framework that bridges natural long-text structures with image components.<n>It preserves global contrastive alignment between images and both summary and long captions.<n>It extends positional embeddings for longer text sequences.
  arXiv  Detail & Related papers  (2025-12-08T03:23:41Z)
• GOAL: Global-local Object Alignment Learning [7.9061560322289335]
  Vision-language models like CLIP have shown impressive capabilities in aligning images and text.<n>They often struggle with lengthy and detailed text descriptions because of their training focus on short and concise captions.<n>We present GOAL, a novel fine-tuning method that enhances CLIP's ability to handle lengthy text.
  arXiv  Detail & Related papers  (2025-03-22T14:27:32Z)
• FLAIR: VLM with Fine-grained Language-informed Image Representations [49.2684130383925]
  FLAIR is an approach that utilizes long and detailed image descriptions to learn localized image embeddings.<n>Our experiments demonstrate the effectiveness of FLAIR trained on 30M image-text pairs in capturing fine-grained visual information.
  arXiv  Detail & Related papers  (2024-12-04T18:56:04Z)
• Contrastive Localized Language-Image Pre-Training [60.4967533101887]
  Contrastive Language-Image Pre-training (CLIP) has been a celebrated method for training vision encoders to generate image/text representations.<n>We propose Contrastive Localized Language-Image Pre-training (CLOC) by complementing CLIP with region-text contrastive loss and modules.<n>CLOC enables high-quality regional embeddings for image region recognition and retrieval tasks.
  arXiv  Detail & Related papers  (2024-10-03T17:56:09Z)
• DIAL: Dense Image-text ALignment for Weakly Supervised Semantic Segmentation [8.422110274212503]
  Weakly supervised semantic segmentation approaches typically rely on class activation maps (CAMs) for initial seed generation. We introduce DALNet, which leverages text embeddings to enhance the comprehensive understanding and precise localization of objects across different levels of granularity. Our approach, in particular, allows for more efficient end-to-end process as a single-stage method.
  arXiv  Detail & Related papers  (2024-09-24T06:51:49Z)
• Selective Vision-Language Subspace Projection for Few-shot CLIP [55.361337202198925]
  We introduce a method called Selective Vision-Language Subspace Projection (SSP) SSP incorporates local image features and utilizes them as a bridge to enhance the alignment between image-text pairs. Our approach entails only training-free matrix calculations and can be seamlessly integrated into advanced CLIP-based few-shot learning frameworks.
  arXiv  Detail & Related papers  (2024-07-24T03:45:35Z)
• AddressCLIP: Empowering Vision-Language Models for City-wide Image Address Localization [57.34659640776723]
  We propose an end-to-end framework named AddressCLIP to solve the problem with more semantics. We have built three datasets from Pittsburgh and San Francisco on different scales specifically for the IAL problem.
  arXiv  Detail & Related papers  (2024-07-11T03:18:53Z)
• SyCoCa: Symmetrizing Contrastive Captioners with Attentive Masking for Multimodal Alignment [11.556516260190737]
  Multimodal alignment between language and vision is the fundamental topic in current vision-language model research. This paper proposes Contrastive Captioners (CoCa) to integrate Contrastive Language-Image Pretraining (CLIP) and Image Caption (IC) into a unified framework.
  arXiv  Detail & Related papers  (2024-01-04T08:42:36Z)
• CLIM: Contrastive Language-Image Mosaic for Region Representation [58.05870131126816]
  Contrastive Language-Image Mosaic (CLIM) is a novel approach for aligning region and text representations. CLIM consistently improves different open-vocabulary object detection methods. It can effectively enhance the region representation of vision-language models.
  arXiv  Detail & Related papers  (2023-12-18T17:39:47Z)
• CRIS: CLIP-Driven Referring Image Segmentation [71.56466057776086]
  We propose an end-to-end CLIP-Driven Referring Image framework (CRIS) CRIS resorts to vision-language decoding and contrastive learning for achieving the text-to-pixel alignment. Our proposed framework significantly outperforms the state-of-the-art performance without any post-processing.
  arXiv  Detail & Related papers  (2021-11-30T07:29:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.