Traj-Transformer: Diffusion Models with Transformer for GPS Trajectory Generation

• URL: http://arxiv.org/abs/2510.06291v1
• Date: Tue, 07 Oct 2025 05:41:09 GMT
• Title: Traj-Transformer: Diffusion Models with Transformer for GPS Trajectory Generation
• Authors: Zhiyang Zhang, Ningcong Chen, Xin Zhang, Yanhua Li, Shen Su, Hui Lu, Jun Luo,
• Abstract summary: We propose Trajectory Transformer, a novel model that employs a transformer backbone for both conditional information embedding and noise prediction.<n>Experiments on two real-world datasets demonstrate that Tray Transformer significantly enhances generation quality and effectively alleviates the issues observed in prior approaches.
• Score: 15.689474391811734
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The widespread use of GPS devices has driven advances in spatiotemporal data mining, enabling machine learning models to simulate human decision making and generate realistic trajectories, addressing both data collection costs and privacy concerns. Recent studies have shown the promise of diffusion models for high-quality trajectory generation. However, most existing methods rely on convolution based architectures (e.g. UNet) to predict noise during the diffusion process, which often results in notable deviations and the loss of fine-grained street-level details due to limited model capacity. In this paper, we propose Trajectory Transformer, a novel model that employs a transformer backbone for both conditional information embedding and noise prediction. We explore two GPS coordinate embedding strategies, location embedding and longitude-latitude embedding, and analyze model performance at different scales. Experiments on two real-world datasets demonstrate that Trajectory Transformer significantly enhances generation quality and effectively alleviates the deviation issues observed in prior approaches.

Related papers

• Detecting Transportation Mode Using Dense Smartphone GPS Trajectories and Transformer Models [11.280640663443826]
  We introduce SpeedTransformer, a novel Transformer-based model that relies solely on speed inputs to infer transportation modes from dense smartphone GPS trajectories.<n>In benchmark experiments, SpeedTransformer outperformed traditional deep learning models, such as the Long Short-Term Memory (LSTM) network.<n>We deployed the model in a real-world experiment, where it consistently outperformed baseline models under complex built environments and high data uncertainty.
  arXiv  Detail & Related papers  (2026-02-27T22:20:29Z)
• Pathlet Variational Auto-Encoder for Robust Trajectory Generation [16.26294619946259]
  Trajectory generation has recently drawn growing interest in privacy-preserving urban mobility studies and location-based service applications.<n>We propose a deep generative model based on the pathlet representation, which encode trajectories with binary vectors associated with a learned dictionary of trajectory segments.<n>Our model can effectively learn data distribution even using noisy data, achieving relative improvements of $35.4%$ and $26.3%$ over strong baselines on two real-world trajectory datasets.
  arXiv  Detail & Related papers  (2025-11-20T06:57:27Z)
• Unveil Benign Overfitting for Transformer in Vision: Training Dynamics, Convergence, and Generalization [88.5582111768376]
  We study the optimization of a Transformer composed of a self-attention layer with softmax followed by a fully connected layer under gradient descent on a certain data distribution model. Our results establish a sharp condition that can distinguish between the small test error phase and the large test error regime, based on the signal-to-noise ratio in the data model.
  arXiv  Detail & Related papers  (2024-09-28T13:24:11Z)
• Hybrid Transformer and Spatial-Temporal Self-Supervised Learning for Long-term Traffic Prediction [1.8531577178922987]
  We propose a model that combines hybrid Transformer and self-supervised learning. The model enhances its adaptive data augmentation by applying data augmentation techniques at the sequence-level of the traffic. We design two self-supervised learning tasks to model the temporal and spatial dependencies, thereby improving the accuracy and ability of the model.
  arXiv  Detail & Related papers  (2024-01-29T06:17:23Z)
• Pre-training on Synthetic Driving Data for Trajectory Prediction [61.520225216107306]
  We propose a pipeline-level solution to mitigate the issue of data scarcity in trajectory forecasting. We adopt HD map augmentation and trajectory synthesis for generating driving data, and then we learn representations by pre-training on them. We conduct extensive experiments to demonstrate the effectiveness of our data expansion and pre-training strategies.
  arXiv  Detail & Related papers  (2023-09-18T19:49:22Z)
• Leveraging the Power of Data Augmentation for Transformer-based Tracking [64.46371987827312]
  We propose two data augmentation methods customized for tracking. First, we optimize existing random cropping via a dynamic search radius mechanism and simulation for boundary samples. Second, we propose a token-level feature mixing augmentation strategy, which enables the model against challenges like background interference.
  arXiv  Detail & Related papers  (2023-09-15T09:18:54Z)
• Emergent Agentic Transformer from Chain of Hindsight Experience [96.56164427726203]
  We show that a simple transformer-based model performs competitively with both temporal-difference and imitation-learning-based approaches. This is the first time that a simple transformer-based model performs competitively with both temporal-difference and imitation-learning-based approaches.
  arXiv  Detail & Related papers  (2023-05-26T00:43:02Z)
• DiffTraj: Generating GPS Trajectory with Diffusion Probabilistic Model [44.490978394267195]
  We propose a spatial-temporal probabilistic model for trajectory generation (DiffTraj) The core idea is to reconstruct and synthesize geographic trajectories from white noise through a reverse trajectory denoising process. Experiments on two real-world datasets show that DiffTraj can be intuitively applied to generate high-fidelity trajectories.
  arXiv  Detail & Related papers  (2023-04-23T08:42:45Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Full Stack Optimization of Transformer Inference: a Survey [58.55475772110702]
  Transformer models achieve superior accuracy across a wide range of applications. The amount of compute and bandwidth required for inference of recent Transformer models is growing at a significant rate. There has been an increased focus on making Transformer models more efficient.
  arXiv  Detail & Related papers  (2023-02-27T18:18:13Z)
• TrAISformer -- A Transformer Network with Sparse Augmented Data Representation and Cross Entropy Loss for AIS-based Vessel Trajectory Prediction [9.281166430457647]
  Vessel trajectory prediction plays a pivotal role in numerous maritime applications and services. forecasting vessel trajectory using AIS data remains challenging, even for modern machine learning techniques. We introduce a discrete, high-dimensional representation of AIS data and a new loss function designed to explicitly address heterogeneous and multimodality. We report experimental results on real, publicly available AIS data. TrAISformer significantly outperforms state-of-the-art methods, with an average prediction performance below 10 nautical miles up to 10 hours.
  arXiv  Detail & Related papers  (2021-09-08T22:44:33Z)
• Transformers Solve the Limited Receptive Field for Monocular Depth Prediction [82.90445525977904]
  We propose TransDepth, an architecture which benefits from both convolutional neural networks and transformers. This is the first paper which applies transformers into pixel-wise prediction problems involving continuous labels.
  arXiv  Detail & Related papers  (2021-03-22T18:00:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.