Spatially-Weighted CLIP for Street-View Geo-localization

• URL: http://arxiv.org/abs/2604.04357v1
• Date: Mon, 06 Apr 2026 02:10:36 GMT
• Title: Spatially-Weighted CLIP for Street-View Geo-localization
• Abstract summary: This paper proposes a novel framework for street-view geo-localization that explicitly incorporates spatial autocorrelation into vision-language contrastive learning.<n>Experiments on a multi-city dataset demonstrate that SW-CLIP significantly improves geo-localization accuracy, reduces long-tail errors, and enhances spatial coherence compared to standard CLIP.
• Score: 5.8048364242505945
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper proposes Spatially-Weighted CLIP (SW-CLIP), a novel framework for street-view geo-localization that explicitly incorporates spatial autocorrelation into vision-language contrastive learning. Unlike conventional CLIP-based methods that treat all non-matching samples as equally negative, SW-CLIP leverages Tobler's First Law of Geography to model geographic relationships through distance-aware soft supervision. Specifically, we introduce a location-as-text representation to encode geographic positions and replace one-hot InfoNCE targets with spatially weighted soft labels derived from geodesic distance. Additionally, a neighborhood-consistency regularization is employed to preserve local spatial structure in the embedding space. Experiments on a multi-city dataset demonstrate that SW-CLIP significantly improves geo-localization accuracy, reduces long-tail errors, and enhances spatial coherence compared to standard CLIP. The results highlight the importance of shifting from semantic alignment to geographic alignment for robust geo-localization and provide a general paradigm for integrating spatial principles into multimodal representation learning.

Related papers

• Location-Aware Language Models via Secondary Embeddings [4.784071359791487]
  We propose a lightweight, model-agnostic approach for injecting geo-spatial awareness into pretrained embeddings.<n>Our method augments input representations with structured geographic signals by combining location names with their corresponding latitude and longitude.<n> Experimental results demonstrate substantial improvements in geo-spatial alignment while maintaining comparable performance on standard NLP benchmarks.
  arXiv  Detail & Related papers  (2026-08-31T22:48:37Z)
• Rethinking Attention Locality in Spiking Transformers [50.00501275379363]
  Spiking Transformers provide a promising paradigm for efficient visual processing with spike-driven computation.<n>Their Softmax-free Spiking Self-Attention (SSA) struggles to establish spatially localized token interactions.<n>We propose Spatially Contiguous Local Attention with Boundary Continuity Pathway (SCLA-BCP)<n>Our approach improves mAP@50 by up to 9.50% on COCO on 2017 and mIoU by up to 3.42% on ADE20K.
  arXiv  Detail & Related papers  (2026-08-09T07:36:17Z)
• UNIGEOCLIP: Unified Geospatial Contrastive Learning [23.04186252685893]
  Co-located geospatial data spans aerial imagery, street-level views, elevation models, text, and geographic coordinates.<n>We introduce UNIGEOCLIP, a massively multimodal contrastive framework to jointly align five complementary geospatial modalities.<n>We also propose a scaled latitude-longitude encoder that improves spatial representation by capturing multi-scale geographic structure.
  arXiv  Detail & Related papers  (2026-04-13T16:14:49Z)
• Global Cross-Modal Geo-Localization: A Million-Scale Dataset and a Physical Consistency Learning Framework [13.268717213871147]
  Cross-modal Geo-localization (CMGL) matches ground-level text descriptions with geo-tagged aerial imagery.<n>We introduce CORE, the first million-scale dataset dedicated to global CMGL.<n>We propose a physical-law-aware network (PLANET) for cross-modal geo-localization.
  arXiv  Detail & Related papers  (2026-03-09T15:27:19Z)
• Reasoning Over Space: Enabling Geographic Reasoning for LLM-Based Generative Next POI Recommendation [8.829656404389178]
  Reasoning Over Space (ROS) is a framework that utilizes geography as a vital decision variable within the reasoning process.<n> ROS introduces a Hierarchical Spatial Semantic ID (SID) that discretizes coarse-to-fine locality and POI semantics into compositional tokens.<n>We further align the model with real world geography via spatial-guided Reinforcement Learning (RL)
  arXiv  Detail & Related papers  (2026-01-08T03:46:03Z)
• Vision-Language Reasoning for Geolocalization: A Reinforcement Learning Approach [41.001581773172695]
  We present Geo-R, a retrieval-free framework that uncovers structured reasoning paths from existing ground-truth coordinates.<n>We propose the Chain of Region, a rule-based hierarchical reasoning paradigm that generates precise, interpretable supervision.<n>Our approach bridges structured geographic reasoning with direct spatial supervision, yielding improved localization accuracy, stronger generalization, and more transparent inference.
  arXiv  Detail & Related papers  (2026-01-01T16:51:41Z)
• CLNet: Cross-View Correspondence Makes a Stronger Geo-Localizationer [48.52152634356309]
  We propose a correspondence-aware feature refinement framework, termed CLNet, that explicitly bridges the semantic and geometric gaps between different views.<n> CLNet decomposes the view alignment process into three learnable and complementary modules.<n>Our proposed CLNet achieves state-of-the-art performance while offering better interpretability and generalizability.
  arXiv  Detail & Related papers  (2025-12-16T16:31:41Z)
• HarmoCLIP: Harmonizing Global and Regional Representations in Contrastive Vision-Language Models [63.87966115136411]
  HarmoCLIP is a novel framework designed to harmonize global and region representations within Contrastive Language-Image Pre-training.<n>To strengthen the representation capability at the local level, our method introduces a novel Region-Language Alignment supervision strategy.
  arXiv  Detail & Related papers  (2025-11-27T16:24:53Z)
• LLM-Aligned Geographic Item Tokenization for Local-Life Recommendation [21.430438767288106]
  LGSID is an LLM-Hierarchical Geographic Item Tokenization Framework for Local-life Recommendation.<n>We introduce a novel G-DPO algorithm that uses pre-trained reward model to inject generalized spatial knowledge and collaborative signals into LLMs.<n>Experiments on real-world Kuaishou industry datasets show that LGSID consistently outperforms state-of-the-art discriminative and generative recommendation models.
  arXiv  Detail & Related papers  (2025-11-18T07:54:32Z)
• Can LLMs Learn to Map the World from Local Descriptions? [50.490593949836146]
  This study investigates whether Large Language Models (LLMs) can construct coherent global spatial cognition.<n> Experiments conducted in a simulated urban environment demonstrate that LLMs exhibit latent representations aligned with real-world spatial distributions.
  arXiv  Detail & Related papers  (2025-05-27T08:22:58Z)
• GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization [61.10806364001535]
  Worldwide Geo-localization aims to pinpoint the precise location of images taken anywhere on Earth. Existing approaches divide the globe into discrete geographic cells, transforming the problem into a classification task. We propose GeoCLIP, a novel CLIP-inspired Image-to-GPS retrieval approach that enforces alignment between the image and its corresponding GPS locations.
  arXiv  Detail & Related papers  (2023-09-27T20:54:56Z)
• Each Part Matters: Local Patterns Facilitate Cross-view Geo-localization [54.00111565818903]
  Cross-view geo-localization is to spot images of the same geographic target from different platforms. Existing methods usually concentrate on mining the fine-grained feature of the geographic target in the image center. We introduce a simple and effective deep neural network, called Local Pattern Network (LPN), to take advantage of contextual information.
  arXiv  Detail & Related papers  (2020-08-26T16:06:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.