TrajPrism: A Multi-Task Benchmark for Language-Grounded Urban Trajectory Understanding

• URL: http://arxiv.org/abs/2605.10782v1
• Date: Mon, 11 May 2026 16:17:13 GMT
• Title: TrajPrism: A Multi-Task Benchmark for Language-Grounded Urban Trajectory Understanding
• Abstract summary: We introduce TrajPrism, a multi-task benchmark for language-trajectory alignment.<n>We construct TrajPrism by pairing real urban trajectories with judge-language annotations generated under a travel-intent taxonomy.<n>The benchmark contains 300K selected trajectories across Porto, San Francisco, and Beijing, yielding 2.1M task instances.
• Score: 9.669733030377632
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Urban mobility is naturally expressed both as trajectories in space and as natural-language descriptions of travel intent, constraints, and preferences. However, prior work rarely evaluates these two modalities together on the same real-world trajectories: trajectory modeling often stays geometry-centric, while language-centric mobility benchmarks frequently target route planning and tool use rather than fine-grained, verifiable alignment between text and the underlying route. We introduce TrajPrism, a multi-task benchmark for language-trajectory alignment that unifies (i) instruction-conditioned trajectory generation, (ii) language-driven semantic trajectory retrieval, and (iii) trajectory captioning, together with an evaluation protocol that measures trajectory fidelity, retrieval quality, and language groundedness. We construct TrajPrism by pairing real urban trajectories with judge-filtered language annotations generated under a four-dimensional travel-intent taxonomy. The benchmark contains 300K selected trajectories across Porto, San Francisco, and Beijing, yielding 2.1M task instances from three instruction variants, three retrieval queries, and one caption per trajectory. We further develop proof-of-concept models for each task: TrajAnchor for instruction-conditioned trajectory generation, TrajFuse for semantic trajectory retrieval, and TrajRap for trajectory captioning. These models instantiate the proposed tasks and show that geometry-only trajectory baselines leave a large gap on our protocol, especially where language is part of the input-output interface. We release TrajPrism with code and a reproducible annotation pipeline that is designed to be portable across cities, given compatible trajectory inputs and map resources.

Related papers

• Trajectory-aware Cross-view Geo-localization with Sequential Observations [14.771498439131953]
  Cross-view-localization matches ground-level observations against satellite imagery.<n>Recent methods show that sequential queries such as video clips yield richer cues than single images.<n>We introduce SeqLoc-VL, a dataset of $sim$39K--text triplets, and TrajLoc, a unified framework capable of processing video clips and route descriptions.
  arXiv  Detail & Related papers  (2026-07-16T22:39:19Z)
• Beyond Waypoints: A Trajectory-Centric Waypointing Paradigm for Vision-Language Navigation [79.12557132788139]
  Vision-Language Navigation in Continuous Environments (VLN-CE) requires agents to follow natural-language instructions while navigating in real-world-like environments.<n>Most VLN-CE approach-es adopt a three-stage framework: a waypoint predictor proposes waypoints, and a navigator selects the best waypoint.<n>We introduce a novel paradigm called Trajectory Waypoint, which grounds each candidate waypoint in an executable trajectory.
  arXiv  Detail & Related papers  (2026-06-05T13:11:39Z)
• InsTraj: Instructing Diffusion Models with Travel Intentions to Generate Real-world Trajectories [58.29205608173041]
  InsTraj is a novel framework that instructs diffusion models to generate high-fidelity trajectories directly from natural language descriptions.<n>We show that InsTraj significantly outperforms state-of-the-art methods in generating trajectories that are realistic, diverse, and semantically faithful to the input instructions.
  arXiv  Detail & Related papers  (2026-04-05T12:56:55Z)
• TrajMamba: An Efficient and Semantic-rich Vehicle Trajectory Pre-training Model [31.21671608884327]
  Vehicle GPS record how vehicles move over time, storing valuable travel semantics, including movement patterns and travel purposes.<n>Learning travel semantics effectively and efficiently is crucial for real-world applications of trajectory data.<n>We propose TrajMamba, a novel approach for efficient and semantically rich vehicle trajectory learning.
  arXiv  Detail & Related papers  (2025-10-20T13:54:50Z)
• TrajSceneLLM: A Multimodal Perspective on Semantic GPS Trajectory Analysis [0.0]
  We propose TrajSceneLLM, a multimodal perspective for enhancing semantic understanding of GPS trajectories.<n>We validate the proposed framework on Travel Mode Identification (TMI), a critical task for analyzing travel choices and understanding mobility behavior.<n>This semantic enhancement promises significant potential for diverse downstream applications and future research in artificial intelligence.
  arXiv  Detail & Related papers  (2025-06-19T15:31:40Z)
• EarthMapper: Visual Autoregressive Models for Controllable Bidirectional Satellite-Map Translation [50.433911327489554]
  We introduce EarthMapper, a novel framework for controllable satellite-map translation.<n>We also contribute CNSatMap, a large-scale dataset comprising 302,132 precisely aligned satellite-map pairs across 38 Chinese cities.<n> experiments on CNSatMap and the New York dataset demonstrate EarthMapper's superior performance.
  arXiv  Detail & Related papers  (2025-04-28T02:41:12Z)
• LangNav: Language as a Perceptual Representation for Navigation [63.90602960822604]
  We explore the use of language as a perceptual representation for vision-and-language navigation (VLN) Our approach uses off-the-shelf vision systems for image captioning and object detection to convert an agent's egocentric panoramic view at each time step into natural language descriptions.
  arXiv  Detail & Related papers  (2023-10-11T20:52:30Z)
• Joint Visual Grounding and Tracking with Natural Language Specification [6.695284124073918]
  Tracking by natural language specification aims to locate the referred target in a sequence based on the natural language description. We propose a joint visual grounding and tracking framework, which reformulates grounding and tracking as a unified task. Our method performs favorably against state-of-the-art algorithms for both tracking and grounding.
  arXiv  Detail & Related papers  (2023-03-21T17:09:03Z)
• BEVBert: Multimodal Map Pre-training for Language-guided Navigation [75.23388288113817]
  We propose a new map-based pre-training paradigm that is spatial-aware for use in vision-and-language navigation (VLN) We build a local metric map to explicitly aggregate incomplete observations and remove duplicates, while modeling navigation dependency in a global topological map. Based on the hybrid map, we devise a pre-training framework to learn a multimodal map representation, which enhances spatial-aware cross-modal reasoning thereby facilitating the language-guided navigation goal.
  arXiv  Detail & Related papers  (2022-12-08T16:27:54Z)
• Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation [87.03299519917019]
  We propose a dual-scale graph transformer (DUET) for joint long-term action planning and fine-grained cross-modal understanding. We build a topological map on-the-fly to enable efficient exploration in global action space. The proposed approach, DUET, significantly outperforms state-of-the-art methods on goal-oriented vision-and-language navigation benchmarks.
  arXiv  Detail & Related papers  (2022-02-23T19:06:53Z)
• Cross-lingual Spoken Language Understanding with Regularized Representation Alignment [71.53159402053392]
  We propose a regularization approach to align word-level and sentence-level representations across languages without any external resource. Experiments on the cross-lingual spoken language understanding task show that our model outperforms current state-of-the-art methods in both few-shot and zero-shot scenarios.
  arXiv  Detail & Related papers  (2020-09-30T08:56:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.