SIREN: Unified Multi-Granularity Semantic Interaction for Multi-Modal Lifelong User Interest Modeling

• URL: http://arxiv.org/abs/2605.25726v1
• Date: Mon, 25 May 2026 11:33:27 GMT
• Title: SIREN: Unified Multi-Granularity Semantic Interaction for Multi-Modal Lifelong User Interest Modeling
• Abstract summary: Industrial recommender systems increasingly leverage lifelong user behavior histories and rich multi-modal content to capture evolving user preferences.<n>We propose SIREN, a unified multi-granularity semantic interaction framework for multi-modal lifelong user interest modeling.<n>From July 2025, SIREN has been fully launched for full-traffic serving in Tencent's advertising platform.
• Score: 21.637289409638655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Industrial recommender systems increasingly leverage lifelong user behavior histories and rich multi-modal content to capture evolving user preferences. However, effectively integrating multi-modal features into lifelong interest modeling remains challenging due to the inherent misalignment between multi-modal and collaborative spaces. Existing paradigms typically rely on separate modeling of multi-modal sequence and behavior sequence, and late fusion to alleviate the modality gap, which results in coarse-grained multi-modal representation and limited integration. In this paper, we propose SIREN, a unified multi-granularity semantic interaction framework for multi-modal lifelong user interest modeling. In the General Search Unit stage, we introduce two alternative retrieval strategies: multi-modal similarity-based soft retrieval for retrieval effectiveness, and Semantic ID (SemID)-based hard retrieval for efficient industrial serving. For the Exact Search Unit stage, we explicitly incorporate target-aware relevance via coarse similarity buckets and fine-grained prefix-encoded SemIDs, enabling unified interaction with collaborative ID features within the target-conditioned transformer architecture. Extensive experiments on the offline dataset demonstrate that SIREN achieves a state-of-the-art GAUC. Online A/B tests further demonstrate consistent GMV gains across multiple production scenarios, including +2.28% in Weixin Moments, +3.87% in Weixin Official Accounts, and +1.61% in Weixin Channels. From July 2025, SIREN has been fully launched for full-traffic serving in Tencent's advertising platform.

Related papers

• MODAL: Multi-Modal Object Re-ID via Model-Driven Sparse Decoupling and Text-Image Differential Filtering [51.99452481869329]
  MODAL is a novel multi-modal object re-identification framework grounded in coupled sparse coding theory and differential suppression principles.<n>A core component of MODAL is a Multi-modal Feature Sparse Decoupling module, developed in a model-driven deep unrolling manner.<n>Benefiting from the principled feature disentanglement, MODAL naturally mitigates performance degradation in incomplete-modality scenarios.<n>Experiments on four datasets demonstrate that MODAL achieves state-of-the-art performance with superior transparency.
  arXiv  Detail & Related papers  (2026-08-15T07:41:10Z)
• GALA: Generative Aligned Learning for Adaptive Multimodal Representation in the Taobao Shangou Recommender System [4.96441528622156]
  We present GALA, a three-stage pipeline that constructs multimodal pretraining data from user behavior and refines it via conversion-based rewards.<n>GALA has been deployed in the production environment at Taobao Shangou, serving over 200 million daily active users.<n>It delivers consistent offline gains of +0.12/+0.20 AUC along with better PCOC metrics.
  arXiv  Detail & Related papers  (2026-07-31T09:32:01Z)
• SiMO: Single-Modality-Operable Multimodal Collaborative Perception [21.057627256755385]
  This paper introduces Single-Modality-Operable Multimodal Collaborative Perception (SiMO)<n>By adopting the proposed Length-Adaptive Multi-Modal Fusion (LAMMA), SiMO can adaptively handle remaining modal features during modal failures.<n>Experiments demonstrate that SiMO effectively aligns multimodal features while simultaneously preserving modality-specific features.
  arXiv  Detail & Related papers  (2026-03-09T11:09:53Z)
• CAMMSR: Category-Guided Attentive Mixture of Experts for Multimodal Sequential Recommendation [23.478610632707728]
  We propose a Category-guided Attentive Mixture of Experts model for Multimodal Sequential Recommendation.<n>At its core, CAMMSR introduces a category-guided attentive mixture of experts module, which learns specialized item representations from multiple perspectives.<n>Experiments on four public datasets demonstrate that CAMMSR consistently outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-04T17:39:35Z)
• MUSE: A Simple Yet Effective Multimodal Search-Based Framework for Lifelong User Interest Modeling [48.18456242206804]
  We present a systematic analysis of how to leverage multimodal signals across both stages of lifelong modeling framework.<n>We propose MUSE, a simple yet effective multimodal search-based framework.<n>MUSE has been deployed in Taobao display advertising system, enabling 100K-length user behavior sequence modeling.
  arXiv  Detail & Related papers  (2025-12-08T06:55:13Z)
• NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching [64.10695425442164]
  We introduce NExT-OMNI, an open-source omnimodal foundation model that achieves unified modeling through discrete flow paradigms.<n>Trained on large-scale interleaved text, image, video, and audio data, NExT-OMNI delivers competitive performance on multimodal generation and understanding benchmarks.<n>To advance further research, we release training details, data protocols, and open-source both the code and model checkpoints.
  arXiv  Detail & Related papers  (2025-10-15T16:25:18Z)
• Progressive Semantic Residual Quantization for Multimodal-Joint Interest Modeling in Music Recommendation [6.790539226766362]
  We propose a novel multimodal recommendation framework with two stages.<n>In the first stage, our method generates modal-specific and modal-joint semantic IDs.<n>In the second stage, to model multimodal interest of users, a Multi-Codebook Cross-Attention network is designed.
  arXiv  Detail & Related papers  (2025-08-28T02:16:57Z)
• MMQ: Multimodal Mixture-of-Quantization Tokenization for Semantic ID Generation and User Behavioral Adaptation [16.81485354427923]
  We propose Multimodal Mixture-of-Quantization (MMQ), a two-stage framework that trains a novel multimodal tokenizer.<n> MMQ unifies multimodal synergy, specificity, and behavioral adaptation, providing a scalable and versatile solution for both generative retrieval and discriminative ranking tasks.
  arXiv  Detail & Related papers  (2025-08-21T06:15:49Z)
• FindRec: Stein-Guided Entropic Flow for Multi-Modal Sequential Recommendation [57.577843653775]
  We propose textbfFindRec (textbfFlexible unified textbfinformation textbfdisentanglement for multi-modal sequential textbfRecommendation)<n>A Stein kernel-based Integrated Information Coordination Module (IICM) theoretically guarantees distribution consistency between multimodal features and ID streams.<n>A cross-modal expert routing mechanism that adaptively filters and combines multimodal features based on their contextual relevance.
  arXiv  Detail & Related papers  (2025-07-07T04:09:45Z)
• U3M: Unbiased Multiscale Modal Fusion Model for Multimodal Semantic Segmentation [63.31007867379312]
  We introduce U3M: An Unbiased Multiscale Modal Fusion Model for Multimodal Semantics. We employ feature fusion at multiple scales to ensure the effective extraction and integration of both global and local features. Experimental results demonstrate that our approach achieves superior performance across multiple datasets.
  arXiv  Detail & Related papers  (2024-05-24T08:58:48Z)
• All in One Framework for Multimodal Re-identification in the Wild [58.380708329455466]
  multimodal learning paradigm for ReID introduced, referred to as All-in-One (AIO) AIO harnesses a frozen pre-trained big model as an encoder, enabling effective multimodal retrieval without additional fine-tuning. Experiments on cross-modal and multimodal ReID reveal that AIO not only adeptly handles various modal data but also excels in challenging contexts.
  arXiv  Detail & Related papers  (2024-05-08T01:04:36Z)
• Unified Multi-modal Unsupervised Representation Learning for Skeleton-based Action Understanding [62.70450216120704]
  Unsupervised pre-training has shown great success in skeleton-based action understanding. We propose a Unified Multimodal Unsupervised Representation Learning framework, called UmURL. UmURL exploits an efficient early-fusion strategy to jointly encode the multi-modal features in a single-stream manner.
  arXiv  Detail & Related papers  (2023-11-06T13:56:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.