Text-like Encoding of Collaborative Information in Large Language Models for Recommendation

• URL: http://arxiv.org/abs/2406.03210v1
• Date: Wed, 05 Jun 2024 12:45:25 GMT
• Title: Text-like Encoding of Collaborative Information in Large Language Models for Recommendation
• Authors: Yang Zhang, Keqin Bao, Ming Yan, Wenjie Wang, Fuli Feng, Xiangnan He,
• Abstract summary: We introduce BinLLM, a novel method to seamlessly integrate collaborative information with Large Language Models for Recommendation (LLMRec) BinLLM converts collaborative embeddings from external models into binary sequences. BinLLM provides options to compress the binary sequence using dot-decimal notation to avoid excessively long lengths.
• Score: 58.87865271693269
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When adapting Large Language Models for Recommendation (LLMRec), it is crucial to integrate collaborative information. Existing methods achieve this by learning collaborative embeddings in LLMs' latent space from scratch or by mapping from external models. However, they fail to represent the information in a text-like format, which may not align optimally with LLMs. To bridge this gap, we introduce BinLLM, a novel LLMRec method that seamlessly integrates collaborative information through text-like encoding. BinLLM converts collaborative embeddings from external models into binary sequences -- a specific text format that LLMs can understand and operate on directly, facilitating the direct usage of collaborative information in text-like format by LLMs. Additionally, BinLLM provides options to compress the binary sequence using dot-decimal notation to avoid excessively long lengths. Extensive experiments validate that BinLLM introduces collaborative information in a manner better aligned with LLMs, resulting in enhanced performance. We release our code at https://github.com/zyang1580/BinLLM.

Related papers

• SelfRACG: Enabling LLMs to Self-Express and Retrieve for Code Generation [63.4105693174085]
  We propose textbfSelfRACG, a novel paradigm that enables large language models (LLMs) to express their information needs to enhance textbfRACG.<n>SelfRACG includes an information need expression module and a two-stage information need-guided training strategy, which encourages LLMs to express their information need.<n>Extensive experiments demonstrate that SelfRACG can retrieve external knowledge that better aligns with the LLM's own information needs, resulting in superior generation performance compared to vanilla RACG.
  arXiv  Detail & Related papers  (2025-07-25T07:42:01Z)
• NL in the Middle: Code Translation with LLMs and Intermediate Representations [66.41928783565795]
  Large language models (LLMs) produce buggy code translations.<n>We consider whether code translation using LLMs can benefit from intermediate representations via natural language (NL) and abstract syntax trees (ASTs)
  arXiv  Detail & Related papers  (2025-07-11T14:29:21Z)
• Large Language Models are Good Relational Learners [55.40941576497973]
  We introduce Rel-LLM, a novel architecture that utilizes a graph neural network (GNN)- based encoder to generate structured relational prompts for large language models (LLMs)<n>Unlike traditional text-based serialization approaches, our method preserves the inherent relational structure of databases while enabling LLMs to process and reason over complex entity relationships.
  arXiv  Detail & Related papers  (2025-06-06T04:07:55Z)
• GEM: Empowering LLM for both Embedding Generation and Language Understanding [11.081595808236239]
  We propose Generative Embedding large language Model (GEM) to generate high-quality text embeddings.<n>Our method inserts new special token(s) into a text body, and generates summarization embedding of the text by manipulating the attention mask.<n>Our results indicate that our approach can empower LLMs with state-of-the-art text embedding capabilities while maintaining their original NLP performance.
  arXiv  Detail & Related papers  (2025-06-04T18:02:07Z)
• Towards a Unified Paradigm: Integrating Recommendation Systems as a New Language in Large Models [33.02146794292383]
  We introduce a new concept, "Integrating Recommendation Systems as a New Language in Large Models" (RSLLM) RSLLM uses a unique prompting method that combines ID-based item embeddings from conventional recommendation models with textual item features. It treats users' sequential behaviors as a distinct language and aligns the ID embeddings with the LLM's input space using a projector.
  arXiv  Detail & Related papers  (2024-12-22T09:08:46Z)
• CoRA: Collaborative Information Perception by Large Language Model's Weights for Recommendation [13.867950651601483]
  Involving collaborative information in Large Language Models (LLMs) is a promising technique for adapting LLMs for recommendation. Existing methods achieve this by concatenating collaborative features with text tokens into a unified sequence input. We propose a new paradigm, textbfCollaborative textbfLoRA, with a collaborative query generator.
  arXiv  Detail & Related papers  (2024-08-20T08:36:59Z)
• Open-domain Implicit Format Control for Large Language Model Generation [52.83173553689678]
  We introduce a novel framework for controlled generation in large language models (LLMs) This study investigates LLMs' capabilities to follow open-domain, one-shot constraints and replicate the format of the example answers. We also develop a dataset collection methodology for supervised fine-tuning that enhances the open-domain format control of LLMs without degrading output quality.
  arXiv  Detail & Related papers  (2024-08-08T11:51:45Z)
• Cool-Fusion: Fuse Large Language Models without Training [73.17551121242602]
  emphCool-Fusion is a method that does not require any type of training like the ensemble approaches. emphCool-Fusion increases accuracy from three strong source LLMs by a significant 8%-17.8%.
  arXiv  Detail & Related papers  (2024-07-29T09:02:19Z)
• CodecLM: Aligning Language Models with Tailored Synthetic Data [51.59223474427153]
  We introduce CodecLM, a framework for adaptively generating high-quality synthetic data for instruction-following abilities. We first encode seed instructions into metadata, which are concise keywords generated on-the-fly to capture the target instruction distribution. We also introduce Self-Rubrics and Contrastive Filtering during decoding to tailor data-efficient samples.
  arXiv  Detail & Related papers  (2024-04-08T21:15:36Z)
• LLM Attributor: Interactive Visual Attribution for LLM Generation [29.116016627864095]
  Python library provides interactive visualizations for training data attribution of large language models. Our library offers a new way to quickly attribute an LLM's text generation to training data points.
  arXiv  Detail & Related papers  (2024-04-01T13:16:34Z)
• Learning to Prompt with Text Only Supervision for Vision-Language Models [107.282881515667]
  One branch of methods adapts CLIP by learning prompts using visual information. An alternative approach resorts to training-free methods by generating class descriptions from large language models. We propose to combine the strengths of both streams by learning prompts using only text data.
  arXiv  Detail & Related papers  (2024-01-04T18:59:49Z)
• Making Large Language Models A Better Foundation For Dense Retrieval [19.38740248464456]
  Dense retrieval needs to learn discriminative text embeddings to represent the semantic relationship between query and document. It may benefit from the using of large language models (LLMs), given LLMs' strong capability on semantic understanding. We propose LLaRA (LLM adapted for dense RetrievAl), which works as a post-hoc adaptation of dense retrieval application.
  arXiv  Detail & Related papers  (2023-12-24T15:10:35Z)
• CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendation [60.2700801392527]
  We introduce CoLLM, an innovative LLMRec methodology that seamlessly incorporates collaborative information into LLMs for recommendation. CoLLM captures collaborative information through an external traditional model and maps it to the input token embedding space of LLM. Extensive experiments validate that CoLLM adeptly integrates collaborative information into LLMs, resulting in enhanced recommendation performance.
  arXiv  Detail & Related papers  (2023-10-30T12:25:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.