Efficient Cold-Start Recommendation via BPE Token-Level Embedding Initialization with LLM

• URL: http://arxiv.org/abs/2509.13179v1
• Date: Tue, 16 Sep 2025 15:32:51 GMT
• Title: Efficient Cold-Start Recommendation via BPE Token-Level Embedding Initialization with LLM
• Authors: Yushang Zhao, Xinyue Han, Qian Leng, Qianyi Sun, Haotian Lyu, Chengrui Zhou,
• Abstract summary: This paper presents an efficient cold-start recommendation strategy based on subword-level representations.<n>We obtain fine-grained token-level vectors that are aligned with the BPE vocabulary.<n>We show that using subword-aware embeddings yields better generalizability and is more interpretable.
• Score: 1.1049570075807806
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The cold-start issue is the challenge when we talk about recommender systems, especially in the case when we do not have the past interaction data of new users or new items. Content-based features or hybrid solutions are common as conventional solutions, but they can only work in a sparse metadata environment with shallow patterns. In this paper, the efficient cold-start recommendation strategy is presented, which is based on the sub word-level representations by applying Byte Pair Encoding (BPE) tokenization and pre-trained Large Language Model (LLM) embedding in the initialization procedure. We obtain fine-grained token-level vectors that are aligned with the BPE vocabulary as opposed to using coarse-grained sentence embeddings. Together, these token embeddings can be used as dense semantic priors on unseen entities, making immediate recommendation performance possible without user-item interaction history. Our mechanism can be compared to collaborative filtering systems and tested over benchmark datasets with stringent cold-start assumptions. Experimental findings show that the given BPE-LLM method achieves higher Recall@k, NDCG@k, and Hit Rate measurements compared to the standard baseline and displays the same capability of sufficient computational performance. Furthermore, we demonstrate that using subword-aware embeddings yields better generalizability and is more interpretable, especially within a multilingual and sparse input setting. The practical application of token-level semantic initialization as a lightweight, but nevertheless effective extension to modern recommender systems in the zero-shot setting is indicated within this work.

Related papers

• Are Large Language Models Really Effective for Training-Free Cold-Start Recommendation? [3.446483216812751]
  This study focuses on training-free recommendation, where no task-specific training is performed.<n>Large language models (LLMs) have recently been explored as a promising solution, and numerous studies have been proposed.<n>We present the first controlled experiments that systematically evaluate these two approaches in the same setting.
  arXiv  Detail & Related papers  (2025-12-15T05:47:07Z)
• Instructional Prompt Optimization for Few-Shot LLM-Based Recommendations on Cold-Start Users [12.794692175339668]
  The cold-start user issue further compromises the effectiveness of recommender systems in limiting access to the historical behavioral information.<n>We introduce a context-conditioned prompt formulation method to optimize instructional prompts on a few-shot large language model (LLM)<n>We show that prompt-based adaptation may be considered one of the ways to address cold-start recommendation issues in LLM-based pipelines.
  arXiv  Detail & Related papers  (2025-09-11T00:13:17Z)
• Tree-Based Text Retrieval via Hierarchical Clustering in RAGFrameworks: Application on Taiwanese Regulations [0.0]
  We propose a hierarchical clustering-based retrieval method that eliminates the need to predefine k.<n>Our approach maintains the accuracy and relevance of system responses while adaptively selecting semantically relevant content.<n>Our framework is simple to implement and easily integrates with existing RAG pipelines, making it a practical solution for real-world applications under limited resources.
  arXiv  Detail & Related papers  (2025-06-16T15:34:29Z)
• HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization [50.27950279695363]
  Many pre-trained language models (PLMs) exhibit suboptimal performance on mid- and low-resource languages.<n>A common strategy to address this is to introduce new tokens specific to the target languages, initialize their embeddings, and apply continual pre-training on target-language data.<n>We propose HYPEROFA, a hypernetwork-based approach for more adaptive token embedding.
  arXiv  Detail & Related papers  (2025-04-21T19:40:32Z)
• Generative Recommendation with Continuous-Token Diffusion [11.23267167046234]
  We propose a novel framework for large language model (LLM)-based recommender systems (RecSys)<n>DeftRec incorporates textbfdenoising ditextbfffusion models to enable LLM-based RecSys to seamlessly support continuous textbftoken as input and target.<n>Given a continuous token as output, recommendations can be easily generated through score-based retrieval.
  arXiv  Detail & Related papers  (2025-04-16T12:01:03Z)
• Training Large Recommendation Models via Graph-Language Token Alignment [53.3142545812349]
  We propose a novel framework to train Large Recommendation models via Graph-Language Token Alignment.<n>By aligning item and user nodes from the interaction graph with pretrained LLM tokens, GLTA effectively leverages the reasoning abilities of LLMs.<n> Furthermore, we introduce Graph-Language Logits Matching (GLLM) to optimize token alignment for end-to-end item prediction.
  arXiv  Detail & Related papers  (2025-02-26T02:19:10Z)
• Unleash LLMs Potential for Recommendation by Coordinating Twin-Tower Dynamic Semantic Token Generator [60.07198935747619]
  We propose Twin-Tower Dynamic Semantic Recommender (T TDS), the first generative RS which adopts dynamic semantic index paradigm. To be more specific, we for the first time contrive a dynamic knowledge fusion framework which integrates a twin-tower semantic token generator into the LLM-based recommender. The proposed T TDS recommender achieves an average improvement of 19.41% in Hit-Rate and 20.84% in NDCG metric, compared with the leading baseline methods.
  arXiv  Detail & Related papers  (2024-09-14T01:45:04Z)
• Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment [104.18002641195442]
  We introduce Self-Augmented Preference Optimization (SAPO), an effective and scalable training paradigm that does not require existing paired data. Building on the self-play concept, which autonomously generates negative responses, we further incorporate an off-policy learning pipeline to enhance data exploration and exploitation.
  arXiv  Detail & Related papers  (2024-05-31T14:21:04Z)
• CELA: Cost-Efficient Language Model Alignment for CTR Prediction [70.65910069412944]
  Click-Through Rate (CTR) prediction holds a paramount position in recommender systems.<n>Recent efforts have sought to mitigate these challenges by integrating Pre-trained Language Models (PLMs)<n>We propose textbfCost-textbfEfficient textbfLanguage Model textbfAlignment (textbfCELA) for CTR prediction.
  arXiv  Detail & Related papers  (2024-05-17T07:43:25Z)
• LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application [54.984348122105516]
  Llm-driven knowlEdge Adaptive RecommeNdation (LEARN) framework synergizes open-world knowledge with collaborative knowledge.<n>We propose an Llm-driven knowlEdge Adaptive RecommeNdation (LEARN) framework that synergizes open-world knowledge with collaborative knowledge.
  arXiv  Detail & Related papers  (2024-05-07T04:00:30Z)
• Large Language Models meet Collaborative Filtering: An Efficient All-round LLM-based Recommender System [19.8986219047121]
  Collaborative filtering recommender systems (CF-RecSys) have shown successive results in enhancing the user experience on social media and e-commerce platforms. Recent strategies have focused on leveraging modality information of user/items based on pre-trained modality encoders and Large Language Models. We propose an efficient All-round LLM-based Recommender system, called A-LLMRec, that excels not only in the cold scenario but also in the warm scenario.
  arXiv  Detail & Related papers  (2024-04-17T13:03:07Z)
• Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferences [33.81337282939615]
  dialog interfaces that allow users to express language-based preferences offer a fundamentally different modality for preference input. Inspired by recent successes of prompting paradigms for large language models (LLMs), we study their use for making recommendations.
  arXiv  Detail & Related papers  (2023-07-26T14:47:15Z)
• Learning to Learn a Cold-start Sequential Recommender [70.5692886883067]
  The cold-start recommendation is an urgent problem in contemporary online applications. We propose a meta-learning based cold-start sequential recommendation framework called metaCSR. metaCSR holds the ability to learn the common patterns from regular users' behaviors.
  arXiv  Detail & Related papers  (2021-10-18T08:11:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.