Recovering Temporal and Geographic Signals from Language Model Embeddings

• URL: http://arxiv.org/abs/2609.05721v1
• Date: Fri, 04 Sep 2026 21:03:06 GMT
• Title: Recovering Temporal and Geographic Signals from Language Model Embeddings
• Abstract summary: We study whether language-model embeddings encode structured real-world information.<n>We use a projection-based method that operates directly on output embeddings.<n>We find that embedding projections recover meaningful chronological and spatial structure.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding whether language-model embeddings encode structured real-world information is important for both representation analysis and information retrieval. We study this question for temporal and geographic signals using a simple projection-based method that operates directly on output embeddings. Given a small set of seed examples, the method defines an axis in embedding space and ranks texts or entities by their projection onto that axis. Our approach is fully black-box and model-agnostic: it requires only embeddings, without access to model weights, internal activations, auxiliary probes, or additional training. This makes it applicable to modern embedding models available only through APIs and provides a lightweight way to analyze whether temporal and spatial dimensions are present in their representation spaces. We apply the method to temporal and geographic datasets and find that embedding projections recover meaningful chronological and spatial structure. These results provide evidence that output embeddings encode signals relevant to time and space, while also offering a practical tool for interpretability and for downstream temporal and geographic information retrieval tasks, such as temporal ordering, geographic ranking, and tagging.

Related papers

• A multi-view contrastive learning framework for spatial embeddings in risk modelling [0.688204255655161]
  spatial data are often unstructured, high-dimensional, and difficult to integrate into predictive models.<n>We propose a novel multi-view contrastive learning framework for generating spatial embeddings.<n>In a case study on French real estate prices, we compare models trained on raw coordinates against those using our spatial embeddings as inputs.
  arXiv  Detail & Related papers  (2025-11-22T07:39:34Z)
• From Time and Place to Preference: LLM-Driven Geo-Temporal Context in Recommendations [3.402237483362839]
  Most recommender systems treat timestamps as numeric or cyclical values, overlooking real-world context such as holidays, events, and seasonal patterns.<n>We propose a scalable framework that uses large language models (LLMs) to generate geo-temporal embeddings from only a timestamp and coarse location.
  arXiv  Detail & Related papers  (2025-10-28T13:57:23Z)
• GT-Loc: Unifying When and Where in Images Through a Joint Embedding Space [48.09196906704634]
  GT-Loc is a novel retrieval-based method that jointly predicts the capture time (hour and month) and geo-location (GPS coordinates) of an image.<n>Our approach employs separate encoders for images, time, and location, aligning their embeddings within a shared high-dimensional feature space.
  arXiv  Detail & Related papers  (2025-07-14T16:54:57Z)
• Geospatial Mechanistic Interpretability of Large Language Models [6.0272491755196045]
  Large Language Models (LLMs) have demonstrated unprecedented capabilities across various natural language processing tasks.<n>Our aim is to advance our understanding of the internal representations that these complex models generate while processing geographical information.
  arXiv  Detail & Related papers  (2025-05-06T09:40:06Z)
• Improving embedding of graphs with missing data by soft manifolds [51.425411400683565]
  The reliability of graph embeddings depends on how much the geometry of the continuous space matches the graph structure. We introduce a new class of manifold, named soft manifold, that can solve this situation. Using soft manifold for graph embedding, we can provide continuous spaces to pursue any task in data analysis over complex datasets.
  arXiv  Detail & Related papers  (2023-11-29T12:48:33Z)
• Reconstructing Spatiotemporal Data with C-VAEs [49.1574468325115]
  Conditional continuous representation of moving regions is commonly used. In this work, we explore the capabilities of Conditional Varitemporal Autoencoder (C-VAE) models to generate realistic representations of regions' evolution.
  arXiv  Detail & Related papers  (2023-07-12T15:34:10Z)
• GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark [56.08664336835741]
  We propose a GeoGraphic Language Understanding Evaluation benchmark, named GeoGLUE. We collect data from open-released geographic resources and introduce six natural language understanding tasks. We pro vide evaluation experiments and analysis of general baselines, indicating the effectiveness and significance of the GeoGLUE benchmark.
  arXiv  Detail & Related papers  (2023-05-11T03:21:56Z)
• TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction [64.63645677568384]
  We introduce a novel saliency prediction model that learns to output saliency maps in sequential time intervals. Our approach locally modulates the saliency predictions by combining the learned temporal maps. Our code will be publicly available on GitHub.
  arXiv  Detail & Related papers  (2023-01-05T22:10:16Z)
• Geo-Adaptive Deep Spatio-Temporal predictive modeling for human mobility [5.864710987890994]
  Deep GA-vLS assumes data to be of fixed and regular shaped tensor shaped and face challenges of handling irregular data. We present a novel geo-aware enabled learning operation based on a novel data structure for dependencies while maintaining the recurrent mechanism.
  arXiv  Detail & Related papers  (2022-11-27T16:51:28Z)
• Geography-Aware Self-Supervised Learning [79.4009241781968]
  We show that due to their different characteristics, a non-trivial gap persists between contrastive and supervised learning on standard benchmarks. We propose novel training methods that exploit the spatially aligned structure of remote sensing data. Our experiments show that our proposed method closes the gap between contrastive and supervised learning on image classification, object detection and semantic segmentation for remote sensing.
  arXiv  Detail & Related papers  (2020-11-19T17:29:13Z)
• Post-Estimation Smoothing: A Simple Baseline for Learning with Side Information [102.18616819054368]
  We propose a post-estimation smoothing operator as a fast and effective method for incorporating structural index data into prediction. Because the smoothing step is separate from the original predictor, it applies to a broad class of machine learning tasks. Our experiments on large scale spatial and temporal datasets highlight the speed and accuracy of post-estimation smoothing in practice.
  arXiv  Detail & Related papers  (2020-03-12T18:04:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.