DenseRec: Revisiting Dense Content Embeddings for Sequential Transformer-based Recommendation

• URL: http://arxiv.org/abs/2508.18442v1
• Date: Mon, 25 Aug 2025 19:47:20 GMT
• Title: DenseRec: Revisiting Dense Content Embeddings for Sequential Transformer-based Recommendation
• Authors: Jan Malte Lichtenberg, Antonio De Candia, Matteo Ruffini,
• Abstract summary: Transformer-based sequential recommenders typically rely solely on learned item ID embeddings.<n>DenseRec is a simple yet effective method that introduces a dual-path embedding approach.<n>In experiments on three real-world datasets, we find DenseRec to consistently outperform an ID-only SASRec baseline.
• Score: 0.24999074238880484
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transformer-based sequential recommenders, such as SASRec or BERT4Rec, typically rely solely on learned item ID embeddings, making them vulnerable to the item cold-start problem, particularly in environments with dynamic item catalogs. While dense content embeddings from pre-trained models offer potential solutions, direct integration into transformer-based recommenders has consistently underperformed compared to ID-only approaches. We revisit this integration challenge and propose DenseRec, a simple yet effective method that introduces a dual-path embedding approach. DenseRec learns a linear projection from the dense embedding space into the ID embedding space during training, enabling seamless generalization to previously unseen items without requiring specialized embedding models or complex infrastructure. In experiments on three real-world datasets, we find DenseRec to consistently outperform an ID-only SASRec baseline, even without additional hyperparameter tuning and while using compact embedding models. Our analysis suggests improvements primarily arise from better sequence representations in the presence of unseen items, positioning DenseRec as a practical and robust solution for cold-start sequential recommendation.

Related papers

• Model Editing for New Document Integration in Generative Information Retrieval [110.90609826290968]
  Generative retrieval (GR) reformulates the Information Retrieval (IR) task as the generation of document identifiers (docIDs)<n>Existing GR models exhibit poor generalization to newly added documents, often failing to generate the correct docIDs.<n>We propose DOME, a novel method that effectively and efficiently adapts GR models to unseen documents.
  arXiv  Detail & Related papers  (2026-03-03T09:13:38Z)
• Towards Efficient and Generalizable Retrieval: Adaptive Semantic Quantization and Residual Knowledge Transfer [11.95059276298165]
  We propose the Anchored Curriculum with Sequential Adaptive Quantization (SA2CRQ) framework.<n>The framework allocates code lengths based on item path entropy, assigning longer, discriminative IDs to head items and shorter, generalizable IDs to tail items.<n>We show that SA2CRQ yields consistent improvements over existing baselines, particularly in cold-start retrieval scenarios.
  arXiv  Detail & Related papers  (2026-02-27T12:39:38Z)
• UniGRec: Unified Generative Recommendation with Soft Identifiers for End-to-End Optimization [20.538589808672963]
  We propose a unified generative recommendation framework, UniGRec.<n>UniGRec addresses training-inference discrepancy, item identifier collapse from codeword usage, and collaborative signal deficiency.<n>Experiments on real-world datasets demonstrate that UniGRec consistently outperforms state-of-the-art baseline methods.
  arXiv  Detail & Related papers  (2026-01-24T12:20:29Z)
• Closing the Performance Gap in Generative Recommenders with Collaborative Tokenization and Efficient Modeling [10.757287948514604]
  We introduce a contrastive tokenization method that integrates collaborative information directly into the learned item representations.<n>We also propose MARIUS, a lightweight, audio-inspired generative model that decouples timeline modeling from item decoding.
  arXiv  Detail & Related papers  (2025-08-12T17:06:55Z)
• eSASRec: Enhancing Transformer-based Recommendations in a Modular Fashion [45.793127165612745]
  Transformer-based models, such as SASRec and BERT4Rec, have become common baselines for sequential recommendations.<n>We identify a very strong model that uses SASRec's training objective, LiGR Transformer layers, and Sampled Softmax Loss.<n>We find that common academic benchmarks show eSASRec to be 23% more effective compared to the most recent state-of-the-art models.
  arXiv  Detail & Related papers  (2025-08-08T16:49:03Z)
• Pre-training Generative Recommender with Multi-Identifier Item Tokenization [78.87007819266957]
  We propose MTGRec to augment token sequence data for Generative Recommender pre-training.<n>Our approach involves two key innovations: multi-identifier item tokenization and curriculum recommender pre-training.<n>Extensive experiments on three public benchmark datasets demonstrate that MTGRec significantly outperforms both traditional and generative recommendation baselines.
  arXiv  Detail & Related papers  (2025-04-06T08:03:03Z)
• Unifying Generative and Dense Retrieval for Sequential Recommendation [37.402860622707244]
  We propose LIGER, a hybrid model that combines the strengths of sequential dense retrieval and generative retrieval.<n> LIGER integrates sequential dense retrieval into generative retrieval, mitigating performance differences and enhancing cold-start item recommendation.<n>This hybrid approach provides insights into the trade-offs between these approaches and demonstrates improvements in efficiency and effectiveness for recommendation systems in small-scale benchmarks.
  arXiv  Detail & Related papers  (2024-11-27T23:36:59Z)
• DimeRec: A Unified Framework for Enhanced Sequential Recommendation via Generative Diffusion Models [39.49215596285211]
  Sequential Recommendation (SR) plays a pivotal role in recommender systems by tailoring recommendations to user preferences based on their non-stationary historical interactions. We propose a novel framework called DimeRec that combines a guidance extraction module (GEM) and a generative diffusion aggregation module (DAM) Our numerical experiments demonstrate that DimeRec significantly outperforms established baseline methods across three publicly available datasets.
  arXiv  Detail & Related papers  (2024-08-22T06:42:09Z)
• Adaptive Cross Batch Normalization for Metric Learning [75.91093210956116]
  Metric learning is a fundamental problem in computer vision. We show that it is equally important to ensure that the accumulated embeddings are up to date. In particular, it is necessary to circumvent the representational drift between the accumulated embeddings and the feature embeddings at the current training iteration.
  arXiv  Detail & Related papers  (2023-03-30T03:22:52Z)
• Boosting Low-Data Instance Segmentation by Unsupervised Pre-training with Saliency Prompt [103.58323875748427]
  This work offers a novel unsupervised pre-training solution for low-data regimes. Inspired by the recent success of the Prompting technique, we introduce a new pre-training method that boosts QEIS models. Experimental results show that our method significantly boosts several QEIS models on three datasets.
  arXiv  Detail & Related papers  (2023-02-02T15:49:03Z)
• DSI++: Updating Transformer Memory with New Documents [95.70264288158766]
  We introduce DSI++, a continual learning challenge for DSI to incrementally index new documents. We show that continual indexing of new documents leads to considerable forgetting of previously indexed documents. We introduce a generative memory to sample pseudo-queries for documents and supplement them during continual indexing to prevent forgetting for the retrieval task.
  arXiv  Detail & Related papers  (2022-12-19T18:59:34Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.