Location-Aware Language Models via Secondary Embeddings

• URL: http://arxiv.org/abs/2609.00454v1
• Date: Mon, 31 Aug 2026 22:48:37 GMT
• Title: Location-Aware Language Models via Secondary Embeddings
• Abstract summary: We propose a lightweight, model-agnostic approach for injecting geo-spatial awareness into pretrained embeddings.<n>Our method augments input representations with structured geographic signals by combining location names with their corresponding latitude and longitude.<n> Experimental results demonstrate substantial improvements in geo-spatial alignment while maintaining comparable performance on standard NLP benchmarks.
• Score: 4.784071359791487
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pretrained transformer-based language models achieve strong performance across a wide range of NLP tasks but remain limited in encoding geo-locational semantics, leading to suboptimal representations of place names and spatial entities. In this work, we propose a lightweight, model-agnostic approach for injecting geo-spatial awareness into pretrained embeddings without modifying the tokenizer or requiring costly retraining. Our method augments input representations with structured geographic signals by combining location names with their corresponding latitude and longitude, and employs a location-focused masking to better align textual representations with real-world spatial relationships. This design allows the model to incorporate geo-spatial context while preserving existing semantic and syntactic knowledge. Experimental results demonstrate substantial improvements in geo-spatial alignment while maintaining comparable performance on standard NLP benchmarks such as GLUE. The method is computationally efficient, requiring only minutes of additional training, and generalizes across multiple model architectures and scales.

Related papers

• FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry [52.232826445242644]
  FoundationGeo bridges relative and metric prediction via spatial calibration and principled data design.<n>Stage 1 learns a high-fidelity, affine-invariant geometry model by initializing with DINOv3.<n>Stage 2 moves beyond global scaling by introducing pixel-wise calibration fields for metric estimation.
  arXiv  Detail & Related papers  (2026-07-13T14:10:01Z)
• ReaGeo: Reasoning-Enhanced End-to-End Geocoding with LLMs [8.089023511080248]
  This paper proposes ReaGeo, an end-to-end geocoding framework based on large language models.<n>The method converts geographic coordinates into geohash sequences, reformulating the coordinate prediction task as a text generation problem.<n>Reinforcement learning with a distance-deviation-based reward is applied to optimize the generation accuracy.
  arXiv  Detail & Related papers  (2026-04-23T07:18:21Z)
• Spatially-Weighted CLIP for Street-View Geo-localization [5.8048364242505945]
  This paper proposes a novel framework for street-view geo-localization that explicitly incorporates spatial autocorrelation into vision-language contrastive learning.<n>Experiments on a multi-city dataset demonstrate that SW-CLIP significantly improves geo-localization accuracy, reduces long-tail errors, and enhances spatial coherence compared to standard CLIP.
  arXiv  Detail & Related papers  (2026-04-06T02:10:36Z)
• Vision-Language Reasoning for Geolocalization: A Reinforcement Learning Approach [41.001581773172695]
  We present Geo-R, a retrieval-free framework that uncovers structured reasoning paths from existing ground-truth coordinates.<n>We propose the Chain of Region, a rule-based hierarchical reasoning paradigm that generates precise, interpretable supervision.<n>Our approach bridges structured geographic reasoning with direct spatial supervision, yielding improved localization accuracy, stronger generalization, and more transparent inference.
  arXiv  Detail & Related papers  (2026-01-01T16:51:41Z)
• Performance and Generalizability Impacts of Incorporating Location Encoders into Deep Learning for Dynamic PM2.5 Estimation [0.39762912548964846]
  We study the effect of geolocation into deep learning for a dynamic and spatially heterogeneous application.<n>We estimate daily surface-level PM2.5 across the contiguous United States using satellite and ground-based observations.<n>Results show that raw coordinates can improve performance within regions by supporting spatial generalization, but can reduce generalizability across regions.
  arXiv  Detail & Related papers  (2025-05-24T02:00:34Z)
• EarthMapper: Visual Autoregressive Models for Controllable Bidirectional Satellite-Map Translation [50.433911327489554]
  We introduce EarthMapper, a novel framework for controllable satellite-map translation.<n>We also contribute CNSatMap, a large-scale dataset comprising 302,132 precisely aligned satellite-map pairs across 38 Chinese cities.<n> experiments on CNSatMap and the New York dataset demonstrate EarthMapper's superior performance.
  arXiv  Detail & Related papers  (2025-04-28T02:41:12Z)
• Swarm Intelligence in Geo-Localization: A Multi-Agent Large Vision-Language Model Collaborative Framework [51.26566634946208]
  We introduce smileGeo, a novel visual geo-localization framework. By inter-agent communication, smileGeo integrates the inherent knowledge of these agents with additional retrieved information. Results show that our approach significantly outperforms current state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-08-21T03:31:30Z)
• Into the Unknown: Generating Geospatial Descriptions for New Environments [18.736071151303726]
  Rendezvous task requires reasoning over allocentric spatial relationships. Using opensource descriptions paired with coordinates (e.g., Wikipedia) provides training data but suffers from limited spatially-oriented text. We propose a large-scale augmentation method for generating high-quality synthetic data for new environments.
  arXiv  Detail & Related papers  (2024-06-28T14:56:21Z)
• Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matching [60.645802236700035]
  Navigating drones through natural language commands remains challenging due to the dearth of accessible multi-modal datasets. We introduce GeoText-1652, a new natural language-guided geo-localization benchmark. This dataset is systematically constructed through an interactive human-computer process.
  arXiv  Detail & Related papers  (2023-11-21T17:52:30Z)
• Geo-Encoder: A Chunk-Argument Bi-Encoder Framework for Chinese Geographic Re-Ranking [61.60169764507917]
  Chinese geographic re-ranking task aims to find the most relevant addresses among retrieved candidates. We propose an innovative framework, namely Geo-Encoder, to more effectively integrate Chinese geographical semantics into re-ranking pipelines.
  arXiv  Detail & Related papers  (2023-09-04T13:44:50Z)
• Global Aggregation then Local Distribution for Scene Parsing [99.1095068574454]
  We show that our approach can be modularized as an end-to-end trainable block and easily plugged into existing semantic segmentation networks. Our approach allows us to build new state of the art on major semantic segmentation benchmarks including Cityscapes, ADE20K, Pascal Context, Camvid and COCO-stuff.
  arXiv  Detail & Related papers  (2021-07-28T03:46:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.