Mixture-of-Experts for Personalized and Semantic-Aware Next Location Prediction

• URL: http://arxiv.org/abs/2505.24597v1
• Date: Fri, 30 May 2025 13:45:19 GMT
• Title: Mixture-of-Experts for Personalized and Semantic-Aware Next Location Prediction
• Authors: Shuai Liu, Ning Cao, Yile Chen, Yue Jiang, Gao Cong,
• Abstract summary: NextLocMoE is a novel framework built upon large language models (LLMs) and structured around a dual-level Mixture-of-Experts (MoE) design.<n>Our architecture comprises two specialized modules: a Location Semantics MoE that operates at the embedding level to encode rich functional semantics of locations, and a Personalized MoE embedded within the Transformer backbone to dynamically adapt to individual user mobility patterns.
• Score: 20.726107072683575
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Next location prediction plays a critical role in understanding human mobility patterns. However, existing approaches face two core limitations: (1) they fall short in capturing the complex, multi-functional semantics of real-world locations; and (2) they lack the capacity to model heterogeneous behavioral dynamics across diverse user groups. To tackle these challenges, we introduce NextLocMoE, a novel framework built upon large language models (LLMs) and structured around a dual-level Mixture-of-Experts (MoE) design. Our architecture comprises two specialized modules: a Location Semantics MoE that operates at the embedding level to encode rich functional semantics of locations, and a Personalized MoE embedded within the Transformer backbone to dynamically adapt to individual user mobility patterns. In addition, we incorporate a history-aware routing mechanism that leverages long-term trajectory data to enhance expert selection and ensure prediction stability. Empirical evaluations across several real-world urban datasets show that NextLocMoE achieves superior performance in terms of predictive accuracy, cross-domain generalization, and interpretability

Related papers

• POIFormer: A Transformer-Based Framework for Accurate and Scalable Point-of-Interest Attribution [3.729614737011418]
  textsfPOIFormer is a novel Transformer-based framework for accurate and efficient POI attribution.<n>textsfPOIFormer enables accurate, efficient attribution in large, noisy mobility datasets.
  arXiv  Detail & Related papers  (2025-07-12T04:37:52Z)
• MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings [75.0617088717528]
  MoCa is a framework for transforming pre-trained VLM backbones into effective bidirectional embedding models.<n>MoCa consistently improves performance across MMEB and ViDoRe-v2 benchmarks, achieving new state-of-the-art results.
  arXiv  Detail & Related papers  (2025-06-29T06:41:00Z)
• Enhancing Large Language Models for Mobility Analytics with Semantic Location Tokenization [29.17336622418242]
  We propose QT-Mob, a novel framework that significantly enhances Large Language Models (LLMs) for mobility analytics.<n> QT-Mob introduces a location tokenization module that learns compact, semantically rich tokens to represent locations.<n>Experiments on three real-world dataset demonstrate the superior performance in both next-location prediction and mobility recovery tasks.
  arXiv  Detail & Related papers  (2025-06-08T02:17:50Z)
• VideoMolmo: Spatio-Temporal Grounding Meets Pointing [66.19964563104385]
  VideoMolmo is a model tailored for fine-grained pointing of video sequences.<n>A novel temporal mask fusion employs SAM2 for bidirectional point propagation.<n>To evaluate the generalization of VideoMolmo, we introduce VPoMolS-temporal, a challenging out-of-distribution benchmark spanning five real-world scenarios.
  arXiv  Detail & Related papers  (2025-06-05T17:59:29Z)
• TrajMoE: Spatially-Aware Mixture of Experts for Unified Human Mobility Modeling [10.338272381612112]
  We propose TrajMoE, a unified and scalable model for cross-city human mobility modeling.<n>TrajMoE addresses two key challenges: (1) inconsistent spatial semantics across cities, and (2) diverse urban mobility patterns.<n>Extensive experiments demonstrate that TrajMoE achieves up to 27% relative improvement over competitive mobility foundation models.
  arXiv  Detail & Related papers  (2025-05-24T12:17:47Z)
• UniSTD: Towards Unified Spatio-Temporal Learning across Diverse Disciplines [64.84631333071728]
  We introduce bfUnistage, a unified Transformer-based framework fortemporal modeling.<n>Our work demonstrates that a task-specific vision-text can build a generalizable model fortemporal learning.<n>We also introduce a temporal module to incorporate temporal dynamics explicitly.
  arXiv  Detail & Related papers  (2025-03-26T17:33:23Z)
• TrajGEOS: Trajectory Graph Enhanced Orientation-based Sequential Network for Mobility Prediction [10.876862361004944]
  We propose a textbfTrajectory textbfGraph textbfEnhanced textbfOrientation-based textbfSequential network (TrajGEOS) for next-location prediction tasks.
  arXiv  Detail & Related papers  (2024-12-26T07:18:38Z)
• A Practitioner's Guide to Continual Multimodal Pretraining [83.63894495064855]
  Multimodal foundation models serve numerous applications at the intersection of vision and language.<n>To keep models updated, research into continual pretraining mainly explores scenarios with either infrequent, indiscriminate updates on large-scale new data, or frequent, sample-level updates.<n>We introduce FoMo-in-Flux, a continual multimodal pretraining benchmark with realistic compute constraints and practical deployment requirements.
  arXiv  Detail & Related papers  (2024-08-26T17:59:01Z)
• AgentMove: A Large Language Model based Agentic Framework for Zero-shot Next Location Prediction [7.007450097312181]
  We introduce AgentMove, a systematic agentic prediction framework to achieve generalized next location prediction.<n>In AgentMove, we first decompose the mobility prediction task and design specific modules to complete them, including spatial-temporal memory for individual mobility pattern mining.<n>Experiments utilizing mobility data from two distinct sources reveal that AgentMove surpasses the leading baseline by 3.33% to 8.57% across 8 out of 12 metrics.
  arXiv  Detail & Related papers  (2024-08-26T02:36:55Z)
• FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts [4.412721048192925]
  We present FedMoE, the efficient personalized Federated Learning framework to address data heterogeneity. FedMoE is composed of two fine-tuning stages. In the first stage, FedMoE simplifies the problem by conducting a search based on observed activation patterns. In the second stage, these submodels are distributed to clients for further training and returned for server aggregating.
  arXiv  Detail & Related papers  (2024-08-21T03:16:12Z)
• DiM-Gesture: Co-Speech Gesture Generation with Adaptive Layer Normalization Mamba-2 framework [2.187990941788468]
  generative model crafted to create highly personalized 3D full-body gestures solely from raw speech audio. Model integrates a Mamba-based fuzzy feature extractor with a non-autoregressive Adaptive Layer Normalization (AdaLN) Mamba-2 diffusion architecture.
  arXiv  Detail & Related papers  (2024-08-01T08:22:47Z)
• Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning [113.89327264634984]
  Few-shot class-incremental learning (FSCIL) confronts the challenge of integrating new classes into a model with minimal training samples. Traditional methods widely adopt static adaptation relying on a fixed parameter space to learn from data that arrive sequentially. We propose a dual selective SSM projector that dynamically adjusts the projection parameters based on the intermediate features for dynamic adaptation.
  arXiv  Detail & Related papers  (2024-07-08T17:09:39Z)
• Unleashing Network Potentials for Semantic Scene Completion [50.95486458217653]
  This paper proposes a novel SSC framework - Adrial Modality Modulation Network (AMMNet) AMMNet introduces two core modules: a cross-modal modulation enabling the interdependence of gradient flows between modalities, and a customized adversarial training scheme leveraging dynamic gradient competition. Extensive experimental results demonstrate that AMMNet outperforms state-of-the-art SSC methods by a large margin.
  arXiv  Detail & Related papers  (2024-03-12T11:48:49Z)
• S2RMs: Spatially Structured Recurrent Modules [105.0377129434636]
  We take a step towards exploiting dynamic structure that are capable of simultaneously exploiting both modular andtemporal structures. We find our models to be robust to the number of available views and better capable of generalization to novel tasks without additional training.
  arXiv  Detail & Related papers  (2020-07-13T17:44:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.