Progressive Collaborative and Semantic Knowledge Fusion for Generative Recommendation

• URL: http://arxiv.org/abs/2502.06269v1
• Date: Mon, 10 Feb 2025 09:08:37 GMT
• Title: Progressive Collaborative and Semantic Knowledge Fusion for Generative Recommendation
• Authors: Longtao Xiao, Haozhao Wang, Cheng Wang, Linfei Ji, Yifan Wang, Jieming Zhu, Zhenhua Dong, Rui Zhang, Ruixuan Li,
• Abstract summary: We propose a progressive collaborative and semantic knowledge fusion model for generative recommendation, named PRORec.<n>In the first stage, we propose a cross-modality knowledge alignment task, which integrates semantic knowledge into collaborative embeddings.<n>In the second stage, we propose an in-modality knowledge distillation task, designed to effectively capture and integrate knowledge from both semantic and collaborative modalities.
• Score: 36.48113842751375
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the recent surge in interest surrounding generative paradigms, generative recommendation has increasingly attracted the attention of researchers in the recommendation community. This paradigm generally consists of two stages. In the first stage, pretrained semantic embeddings or collaborative ID embeddings are quantized to create item codes, aiming to capture and preserve rich semantic or collaborative knowledge within these codes. The second stage involves utilizing these discrete codes to perform an autoregressive sequence generation task. Existing methods often either overlook collaborative or semantic knowledge, or combine the two roughly. In this paper, we observe that naively concatenating representations from semantic and collaborative modality leads to a semantic domination issue, where the resulting representation is overly influenced by semantic information, effectively overshadowing the collaborative representation. Consequently, downstream recommendation tasks fail to fully exploit the knowledge from both modalities, resulting in suboptimal performance. To address this, we propose a progressive collaborative and semantic knowledge fusion model for generative recommendation, named PRORec, which integrates semantic and collaborative knowledge with a unified code through a two-stage framework. Specifically, in the first stage, we propose a cross-modality knowledge alignment task, which integrates semantic knowledge into collaborative embeddings, enhancing their representational capability. In the second stage, we propose an in-modality knowledge distillation task, designed to effectively capture and integrate knowledge from both semantic and collaborative modalities. Extensive experiments on three widely used benchmarks validate the effectiveness of our approach, demonstrating its superiority compared to existing methods.

Related papers

• COHESION: Composite Graph Convolutional Network with Dual-Stage Fusion for Multimodal Recommendation [26.169114011402232]
  Two key processes in multimodal recommendations are modality fusion and representation learning. We introduce a COmposite grapH convolutional nEtwork with dual-stage fuSION for the multimodal recommendation, named COHESION.
  arXiv  Detail & Related papers  (2025-04-06T11:42:49Z)
• Bridging Textual-Collaborative Gap through Semantic Codes for Sequential Recommendation [91.13055384151897]
  CoCoRec is a novel Code-based textual and Collaborative semantic fusion method for sequential Recommendation. We generate fine-grained semantic codes from multi-view text embeddings through vector quantization techniques. In order to further enhance the fusion of textual and collaborative semantics, we introduce an optimization strategy.
  arXiv  Detail & Related papers  (2025-03-15T15:54:44Z)
• EAGER: Two-Stream Generative Recommender with Behavior-Semantic Collaboration [63.112790050749695]
  We introduce EAGER, a novel generative recommendation framework that seamlessly integrates both behavioral and semantic information. We validate the effectiveness of EAGER on four public benchmarks, demonstrating its superior performance compared to existing methods.
  arXiv  Detail & Related papers  (2024-06-20T06:21:56Z)
• Concept Matching with Agent for Out-of-Distribution Detection [19.407364109506904]
  We propose a new method that integrates the agent paradigm into out-of-distribution (OOD) detection task.<n>Our proposed method, Concept Matching with Agent (CMA), employs neutral prompts as agents to augment the CLIP-based OOD detection process.<n>Our extensive experimental results showcase the superior performance of CMA over both zero-shot and training-required methods.
  arXiv  Detail & Related papers  (2024-05-27T02:27:28Z)
• LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application [54.984348122105516]
  Llm-driven knowlEdge Adaptive RecommeNdation (LEARN) framework synergizes open-world knowledge with collaborative knowledge.<n>We propose an Llm-driven knowlEdge Adaptive RecommeNdation (LEARN) framework that synergizes open-world knowledge with collaborative knowledge.
  arXiv  Detail & Related papers  (2024-05-07T04:00:30Z)
• Co-guiding for Multi-intent Spoken Language Understanding [53.30511968323911]
  We propose a novel model termed Co-guiding Net, which implements a two-stage framework achieving the mutual guidances between the two tasks. For the first stage, we propose single-task supervised contrastive learning, and for the second stage, we propose co-guiding supervised contrastive learning. Experiment results on multi-intent SLU show that our model outperforms existing models by a large margin.
  arXiv  Detail & Related papers  (2023-11-22T08:06:22Z)
• REKnow: Enhanced Knowledge for Joint Entity and Relation Extraction [30.829001748700637]
  Relation extraction is a challenging task that aims to extract all hidden relational facts from the text. There is no unified framework that works well under various relation extraction settings. We propose a knowledge-enhanced generative model to mitigate these two issues. Our model achieves superior performance on multiple benchmarks and settings, including WebNLG, NYT10, and TACRED.
  arXiv  Detail & Related papers  (2022-06-10T13:59:38Z)
• Human-Centered Prior-Guided and Task-Dependent Multi-Task Representation Learning for Action Recognition Pre-Training [8.571437792425417]
  We propose a novel action recognition pre-training framework, which exploits human-centered prior knowledge that generates more informative representation. Specifically, we distill knowledge from a human parsing model to enrich the semantic capability of representation. In addition, we combine knowledge distillation with contrastive learning to constitute a task-dependent multi-task framework.
  arXiv  Detail & Related papers  (2022-04-27T06:51:31Z)
• PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval [87.68667887072324]
  We propose a novel approach that leverages query-centric and PAssage-centric sImilarity Relations (called PAIR) for dense passage retrieval. To implement our approach, we make three major technical contributions by introducing formal formulations of the two kinds of similarity relations. Our approach significantly outperforms previous state-of-the-art models on both MSMARCO and Natural Questions datasets.
  arXiv  Detail & Related papers  (2021-08-13T02:07:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.