Classical Feature Embeddings Help in BERT-Based Human Mobility Prediction

• URL: http://arxiv.org/abs/2510.20275v1
• Date: Thu, 23 Oct 2025 06:59:58 GMT
• Title: Classical Feature Embeddings Help in BERT-Based Human Mobility Prediction
• Authors: Yunzhi Liu, Haokai Tan, Rushi Kanjaria, Lihuan Li, Flora D. Salim,
• Abstract summary: We propose STaBERT (Semantic-Temporal aware BERT), which integrates both POI and temporal information at each location to construct a unified, semantically enriched representation of mobility.<n> Experimental results show that STaBERT significantly improves prediction accuracy.
• Score: 10.415834763540735
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human mobility forecasting is crucial for disaster relief, city planning, and public health. However, existing models either only model location sequences or include time information merely as auxiliary input, thereby failing to leverage the rich semantic context provided by points of interest (POIs). To address this, we enrich a BERT-based mobility model with derived temporal descriptors and POI embeddings to better capture the semantics underlying human movement. We propose STaBERT (Semantic-Temporal aware BERT), which integrates both POI and temporal information at each location to construct a unified, semantically enriched representation of mobility. Experimental results show that STaBERT significantly improves prediction accuracy: for single-city prediction, the GEO-BLEU score improved from 0.34 to 0.75; for multi-city prediction, from 0.34 to 0.56.

Related papers

• Learning Multi-Modal Mobility Dynamics for Generalized Next Location Recommendation [51.00494428978262]
  We leverage multi-modal spatial-temporal knowledge to characterize mobility dynamics for the location recommendation task.<n>First, we construct a unified spatial-temporal relational graph (STRG) for multi-modal representation.<n>Second, we design a gating mechanism to fuse spatial-temporal graph representations of different modalities.
  arXiv  Detail & Related papers  (2025-12-27T14:23:04Z)
• GSTM-HMU: Generative Spatio-Temporal Modeling for Human Mobility Understanding [12.79351579779076]
  We introduce GSTM-HMU, a generative temporal-temporal framework to advance mobility analysis.<n>We conduct experiments on four widely used real-world datasets, including Gowalla, WeePlace, Brightkite, and FourSquare.
  arXiv  Detail & Related papers  (2025-09-23T15:20:38Z)
• Predicting Human Mobility in Disasters via LLM-Enhanced Cross-City Learning [11.567067262043544]
  DisasterMobLLM is a mobility prediction framework for disaster scenarios.<n>It can be integrated into existing deep mobility prediction methods.<n>It can achieve a 32.8% improvement in terms of Acc@1 and a 35.0% improvement in terms of the F1-score of predicting immobility.
  arXiv  Detail & Related papers  (2025-07-26T01:45:27Z)
• ALOcc: Adaptive Lifting-Based 3D Semantic Occupancy and Cost Volume-Based Flow Predictions [91.55655961014027]
  3D semantic occupancy and flow prediction are fundamental to understanding scene scene.<n>This paper proposes a vision-based framework with three targeted improvements.<n>Our purely convolutional architecture establishes new SOTA performance on multiple benchmarks for both semantic occupancy and joint semantic-flow prediction.
  arXiv  Detail & Related papers  (2024-11-12T11:32:56Z)
• Physics-guided Active Sample Reweighting for Urban Flow Prediction [75.24539704456791]
  Urban flow prediction is a nuanced-temporal modeling that estimates the throughput of transportation services like buses, taxis and ride-driven models. Some recent prediction solutions bring remedies with the notion of physics-guided machine learning (PGML) We develop a atized physics-guided network (PN), and propose a data-aware framework Physics-guided Active Sample Reweighting (P-GASR)
  arXiv  Detail & Related papers  (2024-07-18T15:44:23Z)
• Towards Generalizable and Interpretable Motion Prediction: A Deep Variational Bayes Approach [54.429396802848224]
  This paper proposes an interpretable generative model for motion prediction with robust generalizability to out-of-distribution cases. For interpretability, the model achieves the target-driven motion prediction by estimating the spatial distribution of long-term destinations. Experiments on motion prediction datasets validate that the fitted model can be interpretable and generalizable.
  arXiv  Detail & Related papers  (2024-03-10T04:16:04Z)
• Prediction-Oriented Bayesian Active Learning [51.426960808684655]
  Expected predictive information gain (EPIG) is an acquisition function that measures information gain in the space of predictions rather than parameters. EPIG leads to stronger predictive performance compared with BALD across a range of datasets and models.
  arXiv  Detail & Related papers  (2023-04-17T10:59:57Z)
• Context-aware multi-head self-attentional neural network model for next location prediction [19.640761373993417]
  We utilize a multi-head self-attentional (A) neural network that learns location patterns from historical location visits. We demonstrate that proposed the model outperforms other state-of-the-art prediction models. We believe that the proposed model is vital for context-aware mobility prediction.
  arXiv  Detail & Related papers  (2022-12-04T23:40:14Z)
• Conditioned Human Trajectory Prediction using Iterative Attention Blocks [70.36888514074022]
  We present a simple yet effective pedestrian trajectory prediction model aimed at pedestrians positions prediction in urban-like environments. Our model is a neural-based architecture that can run several layers of attention blocks and transformers in an iterative sequential fashion. We show that without explicit introduction of social masks, dynamical models, social pooling layers, or complicated graph-like structures, it is possible to produce on par results with SoTA models.
  arXiv  Detail & Related papers  (2022-06-29T07:49:48Z)
• MobTCast: Leveraging Auxiliary Trajectory Forecasting for Human Mobility Prediction [5.911865723926626]
  We propose MobTCast, a Transformer-based context-aware network for mobility prediction. We explore the influence of four types of context in the mobility prediction: temporal, semantic, social and geographical contexts. In our experimental results, MobTCast outperforms other state-of-the-art next POI prediction methods.
  arXiv  Detail & Related papers  (2021-09-30T09:42:15Z)
• Individual Mobility Prediction: An Interpretable Activity-based Hidden Markov Approach [6.1938383008964495]
  This study develops an activity-based modeling framework for individual mobility prediction. We show that the proposed model can achieve similar prediction performance as the state-of-the-art long-term short-term memory (LSTM) model.
  arXiv  Detail & Related papers  (2021-01-11T16:11:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.