Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning

• URL: http://arxiv.org/abs/2212.03760v5
• Date: Sat, 13 May 2023 08:16:21 GMT
• Title: Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning
• Authors: Kyuyong Shin, Hanock Kwak, Wonjae Kim, Jisu Jeong, Seungjae Jung, Kyung-Min Kim, Jung-Woo Ha, Sang-Woo Lee
• Abstract summary: We show that language modeling applied directly to task-specific user histories achieves excellent results on diverse recommendation tasks. We further demonstrate that our approach can provide promising transfer learning capabilities for a broad spectrum of real-world recommender systems.
• Score: 23.119223101680976
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent studies have proposed unified user modeling frameworks that leverage user behavior data from various applications. Many of them benefit from utilizing users' behavior sequences as plain texts, representing rich information in any domain or system without losing generality. Hence, a question arises: Can language modeling for user history corpus help improve recommender systems? While its versatile usability has been widely investigated in many domains, its applications to recommender systems still remain underexplored. We show that language modeling applied directly to task-specific user histories achieves excellent results on diverse recommendation tasks. Also, leveraging additional task-agnostic user histories delivers significant performance benefits. We further demonstrate that our approach can provide promising transfer learning capabilities for a broad spectrum of real-world recommender systems, even on unseen domains and services.

Related papers

• Creating General User Models from Computer Use [62.91116265732001]
  This paper presents an architecture for a general user model (GUM) that learns about you by observing any interaction you have with your computer.<n>The GUM takes as input any unstructured observation of a user (e.g., device screenshots) and constructs confidence-weighted propositions that capture user knowledge and preferences.
  arXiv  Detail & Related papers  (2025-05-16T04:00:31Z)
• HistLLM: A Unified Framework for LLM-Based Multimodal Recommendation with User History Encoding and Compression [33.34435467588446]
  HistLLM is an innovative framework that integrates textual and visual features through a User History. Module (UHEM), compressing user history interactions into a single token representation. Extensive experiments demonstrate the effectiveness and efficiency of our proposed mechanism.
  arXiv  Detail & Related papers  (2025-04-14T12:01:11Z)
• Large Language Model Empowered Recommendation Meets All-domain Continual Pre-Training [60.38082979765664]
  CPRec is an All-domain Continual Pre-Training framework for Recommendation. It holistically align LLMs with universal user behaviors through the continual pre-training paradigm. We conduct experiments on five real-world datasets from two distinct platforms.
  arXiv  Detail & Related papers  (2025-04-11T20:01:25Z)
• RALLRec: Improving Retrieval Augmented Large Language Model Recommendation with Representation Learning [24.28601381739682]
  Large Language Models (LLMs) have been integrated into recommendation systems to enhance user behavior comprehension. Existing RAG methods rely primarily on textual semantics and often fail to incorporate the most relevant items. We propose Representation learning for retrieval-Augmented Large Language model Recommendation (RALLRec)
  arXiv  Detail & Related papers  (2025-02-10T02:15:12Z)
• Generalized User Representations for Transfer Learning [6.953653891411339]
  We present a novel framework for user representation in large-scale recommender systems. Our approach employs a two-stage methodology combining representation learning and transfer learning. We show how the proposed framework can significantly reduce infrastructure costs compared to alternative approaches.
  arXiv  Detail & Related papers  (2024-03-01T15:05:21Z)
• User Embedding Model for Personalized Language Prompting [9.472634942498859]
  We introduce a new User Embedding Module (UEM) that efficiently processes user history in free-form text by compressing and representing them as embeddings. Our experiments demonstrate the superior capability of this approach in handling significantly longer histories. The main contribution of this research is to demonstrate the ability to bias language models with user signals represented as embeddings.
  arXiv  Detail & Related papers  (2024-01-10T00:35:52Z)
• Explore-Instruct: Enhancing Domain-Specific Instruction Coverage through Active Exploration [64.58185031596169]
  Explore-Instruct is a novel approach to enhance the data coverage to be used in domain-specific instruction-tuning. Our data-centric analysis validates the effectiveness of this proposed approach in improving domain-specific instruction coverage. Our findings offer a promising opportunity to improve instruction coverage, especially in domain-specific contexts.
  arXiv  Detail & Related papers  (2023-10-13T15:03:15Z)
• Reformulating Sequential Recommendation: Learning Dynamic User Interest with Content-enriched Language Modeling [18.297332953450514]
  We propose LANCER, which leverages the semantic understanding capabilities of pre-trained language models to generate personalized recommendations. Our approach bridges the gap between language models and recommender systems, resulting in more human-like recommendations.
  arXiv  Detail & Related papers  (2023-09-19T08:54:47Z)
• Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations [63.19448893196642]
  We present a novel framework that harnesses the rich contextual information and semantic representations provided by large language models to analyze behavior graphs. By leveraging this capability, our framework enables personalized and accurate job recommendations for individual users.
  arXiv  Detail & Related papers  (2023-07-10T11:29:41Z)
• Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach [83.62750225073341]
  We consider recommendation as instruction following by large language models (LLMs) We first design a general instruction format for describing the preference, intention, task form and context of a user in natural language. Then we manually design 39 instruction templates and automatically generate a large amount of user-personalized instruction data.
  arXiv  Detail & Related papers  (2023-05-11T17:39:07Z)
• Knowledge-Enhanced Hierarchical Graph Transformer Network for Multi-Behavior Recommendation [56.12499090935242]
  This work proposes a Knowledge-Enhanced Hierarchical Graph Transformer Network (KHGT) to investigate multi-typed interactive patterns between users and items in recommender systems. KHGT is built upon a graph-structured neural architecture to capture type-specific behavior characteristics. We show that KHGT consistently outperforms many state-of-the-art recommendation methods across various evaluation settings.
  arXiv  Detail & Related papers  (2021-10-08T09:44:00Z)
• Generative Adversarial Reward Learning for Generalized Behavior Tendency Inference [71.11416263370823]
  We propose a generative inverse reinforcement learning for user behavioral preference modelling. Our model can automatically learn the rewards from user's actions based on discriminative actor-critic network and Wasserstein GAN.
  arXiv  Detail & Related papers  (2021-05-03T13:14:25Z)
• Explainable Recommender Systems via Resolving Learning Representations [57.24565012731325]
  Explanations could help improve user experience and discover system defects. We propose a novel explainable recommendation model through improving the transparency of the representation learning process.
  arXiv  Detail & Related papers  (2020-08-21T05:30:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.