Do We Really Need Specialization? Evaluating Generalist Text Embeddings for Zero-Shot Recommendation and Search

• URL: http://arxiv.org/abs/2507.05006v2
• Date: Tue, 08 Jul 2025 06:16:29 GMT
• Title: Do We Really Need Specialization? Evaluating Generalist Text Embeddings for Zero-Shot Recommendation and Search
• Authors: Matteo Attimonelli, Alessandro De Bellis, Claudio Pomo, Dietmar Jannach, Eugenio Di Sciascio, Tommaso Di Noia,
• Abstract summary: Generalist Text Embedding Models (GTEs) are pre-trained on large-scale corpora.<n>We show that GTEs can guarantee strong zero-shot performance without specialized adaptation.<n>We also show that compressing embedding dimensions by focusing on the most informative directions effectively reduces noise.
• Score: 51.30225948318648
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Pre-trained language models (PLMs) are widely used to derive semantic representations from item metadata in recommendation and search. In sequential recommendation, PLMs enhance ID-based embeddings through textual metadata, while in product search, they align item characteristics with user intent. Recent studies suggest task and domain-specific fine-tuning are needed to improve representational power. This paper challenges this assumption, showing that Generalist Text Embedding Models (GTEs), pre-trained on large-scale corpora, can guarantee strong zero-shot performance without specialized adaptation. Our experiments demonstrate that GTEs outperform traditional and fine-tuned models in both sequential recommendation and product search. We attribute this to a superior representational power, as they distribute features more evenly across the embedding space. Finally, we show that compressing embedding dimensions by focusing on the most informative directions (e.g., via PCA) effectively reduces noise and improves the performance of specialized models. To ensure reproducibility, we provide our repository at https://split.to/gte4ps.

Related papers

• End-to-End Personalization: Unifying Recommender Systems with Large Language Models [0.0]
  We propose a novel hybrid recommendation framework that combines Graph Attention Networks (GATs) with Large Language Models (LLMs)<n>LLMs are first used to enrich user and item representations by generating semantically meaningful profiles based on metadata such as titles, genres, and overviews.<n>We evaluate our model on benchmark datasets, including MovieLens 100k and 1M, where it consistently outperforms strong baselines.
  arXiv  Detail & Related papers  (2025-08-02T22:46:50Z)
• Towards Bridging Review Sparsity in Recommendation with Textual Edge Graph Representation [28.893058826607735]
  We propose a unified framework that imputes missing reviews by jointly modeling semantic and structural signals.<n>Experiments on the Amazon and Goodreads datasets show that TWISTER consistently outperforms traditional numeric, graph-based, and LLM baselines.<n>In summary, TWISTER generates reviews that are more helpful, authentic, and specific, while smoothing structural signals for improved recommendations.
  arXiv  Detail & Related papers  (2025-08-02T00:53:40Z)
• Scaling Transformers for Discriminative Recommendation via Generative Pretraining [15.796591192359044]
  We propose a framework named GPSD (textbfGenerative textbfPretraining for textbfScalable textbfDiscriminative Recommendation) to address the overfitting issue.<n>Extensive experiments conducted on both industrial-scale and publicly available datasets demonstrate the superior performance of GPSD.
  arXiv  Detail & Related papers  (2025-06-04T08:31:33Z)
• Adapting General-Purpose Embedding Models to Private Datasets Using Keyword-based Retrieval [19.57735892785756]
  BMEmbed is a novel method for adapting general-purpose text embedding models to private datasets.<n>We construct supervisory signals from the ranking of keyword-based retrieval results to facilitate model adaptation.<n>We evaluate BMEmbed across a range of domains, datasets, and models, showing consistent improvements in retrieval performance.
  arXiv  Detail & Related papers  (2025-05-31T03:06:09Z)
• Inductive Generative Recommendation via Retrieval-based Speculation [26.70518822003545]
  Generative recommendation (GR) is an emerging paradigm that tokenizes items into discrete tokens and learns to autoregressively generate the next tokens as predictions. In this paper, we propose SpecGR, a plug-and-play framework that enables GR models to recommend new items in an inductive setting.
  arXiv  Detail & Related papers  (2024-10-03T19:32:32Z)
• Text Matching Improves Sequential Recommendation by Reducing Popularity Biases [48.272381505993366]
  TASTE verbalizes items and user-item interactions using identifiers and attributes of items. Our experiments show that TASTE outperforms the state-of-the-art methods on widely used sequential recommendation datasets.
  arXiv  Detail & Related papers  (2023-08-27T07:44:33Z)
• Enhancing Black-Box Few-Shot Text Classification with Prompt-Based Data Augmentation [42.05617728412819]
  We show how to optimize few-shot text classification without accessing the gradients of the large-scale language models. Our approach, dubbed BT-Classifier, significantly outperforms state-of-the-art black-box few-shot learners.
  arXiv  Detail & Related papers  (2023-05-23T07:54:34Z)
• Attentive Graph-based Text-aware Preference Modeling for Top-N Recommendation [2.3991565023534083]
  We propose a new model named Attentive Graph-based Text-aware Recommendation Model (AGTM) In this work, we aim to further improve top-N recommendation by effectively modeling both item textual content and high-order connectivity in user-item graph.
  arXiv  Detail & Related papers  (2023-05-22T12:32:06Z)
• POUF: Prompt-oriented unsupervised fine-tuning for large pre-trained models [62.23255433487586]
  We propose an unsupervised fine-tuning framework to fine-tune the model or prompt on the unlabeled target data. We demonstrate how to apply our method to both language-augmented vision and masked-language models by aligning the discrete distributions extracted from the prompts and target data.
  arXiv  Detail & Related papers  (2023-04-29T22:05:22Z)
• Studying How to Efficiently and Effectively Guide Models with Explanations [52.498055901649025]
  'Model guidance' is the idea of regularizing the models' explanations to ensure that they are "right for the right reasons" We conduct an in-depth evaluation across various loss functions, attribution methods, models, and 'guidance depths' on the PASCAL VOC 2007 and MS COCO 2014 datasets. Specifically, we guide the models via bounding box annotations, which are much cheaper to obtain than the commonly used segmentation masks.
  arXiv  Detail & Related papers  (2023-03-21T15:34:50Z)
• Teacher Guided Training: An Efficient Framework for Knowledge Transfer [86.6784627427194]
  We propose the teacher-guided training (TGT) framework for training a high-quality compact model. TGT exploits the fact that the teacher has acquired a good representation of the underlying data domain. We find that TGT can improve accuracy on several image classification benchmarks and a range of text classification and retrieval tasks.
  arXiv  Detail & Related papers  (2022-08-14T10:33:58Z)
• Fine-grained Retrieval Prompt Tuning [149.9071858259279]
  Fine-grained Retrieval Prompt Tuning steers a frozen pre-trained model to perform the fine-grained retrieval task from the perspectives of sample prompt and feature adaptation. Our FRPT with fewer learnable parameters achieves the state-of-the-art performance on three widely-used fine-grained datasets.
  arXiv  Detail & Related papers  (2022-07-29T04:10:04Z)
• DAGA: Data Augmentation with a Generation Approach for Low-resource Tagging Tasks [88.62288327934499]
  We propose a novel augmentation method with language models trained on the linearized labeled sentences. Our method is applicable to both supervised and semi-supervised settings.
  arXiv  Detail & Related papers  (2020-11-03T07:49:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.