RecoBERT: A Catalog Language Model for Text-Based Recommendations

• URL: http://arxiv.org/abs/2009.13292v1
• Date: Fri, 25 Sep 2020 14:23:38 GMT
• Title: RecoBERT: A Catalog Language Model for Text-Based Recommendations
• Abstract summary: RecoBERT is a BERT-based approach for learning catalog-specialized language models for text-based item recommendations. We introduce a new language understanding task for wine recommendations using similarities based on professional wine reviews.
• Score: 32.40792615018446
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Language models that utilize extensive self-supervised pre-training from unlabeled text, have recently shown to significantly advance the state-of-the-art performance in a variety of language understanding tasks. However, it is yet unclear if and how these recent models can be harnessed for conducting text-based recommendations. In this work, we introduce RecoBERT, a BERT-based approach for learning catalog-specialized language models for text-based item recommendations. We suggest novel training and inference procedures for scoring similarities between pairs of items, that don't require item similarity labels. Both the training and the inference techniques were designed to utilize the unlabeled structure of textual catalogs, and minimize the discrepancy between them. By incorporating four scores during inference, RecoBERT can infer text-based item-to-item similarities more accurately than other techniques. In addition, we introduce a new language understanding task for wine recommendations using similarities based on professional wine reviews. As an additional contribution, we publish annotated recommendations dataset crafted by human wine experts. Finally, we evaluate RecoBERT and compare it to various state-of-the-art NLP models on wine and fashion recommendations tasks.

Related papers

• Classifying the Unknown: In-Context Learning for Open-Vocabulary Text and Symbol Recognition [3.667678728817253]
  We introduce Rosetta, a multimodal model that leverages Multimodal In-Context Learning (MICL) to classify sequences of novel script patterns in documents. A key strength of our method is the use of a Context-Aware Tokenizer (CAT), which enables open-vocabulary classification. As a result, it unlocks applications such as the recognition of new alphabets and languages.
  arXiv  Detail & Related papers  (2025-04-09T12:58:25Z)
• Enhancing Recommender Systems Using Textual Embeddings from Pre-trained Language Models [2.3020018305241337]
  In this paper, we explore enhancing recommender systems using textual embeddings from pre-trained language models. Our experiments demonstrate that this approach significantly improves recommendation accuracy and relevance, resulting in more personalized and context-aware recommendations.
  arXiv  Detail & Related papers  (2025-03-24T09:03:12Z)
• FLARE: Fusing Language Models and Collaborative Architectures for Recommender Enhancement [0.7372033475418547]
  Flare is a novel hybrid sequence recommender that integrates a language model with a collaborative filtering model using a Perceiver network. We revisit the often-used Bert4Rec baseline and show that Bert4Rec significantly outperforms previously reported numbers. This paper also showcases Flare's inherent ability to support critiquing, enabling users to provide feedback and refine recommendations.
  arXiv  Detail & Related papers  (2024-09-18T04:43:41Z)
• MACT: Model-Agnostic Cross-Lingual Training for Discourse Representation Structure Parsing [4.536003573070846]
  We introduce a cross-lingual training strategy for semantic representation parsing models. It exploits the alignments between languages encoded in pre-trained language models. Experiments show significant improvements in DRS clause and graph parsing in English, German, Italian and Dutch.
  arXiv  Detail & Related papers  (2024-06-03T07:02:57Z)
• A Novel Cartography-Based Curriculum Learning Method Applied on RoNLI: The First Romanian Natural Language Inference Corpus [71.77214818319054]
  Natural language inference is a proxy for natural language understanding. There is no publicly available NLI corpus for the Romanian language. We introduce the first Romanian NLI corpus (RoNLI) comprising 58K training sentence pairs.
  arXiv  Detail & Related papers  (2024-05-20T08:41:15Z)
• Knowledge Graphs and Pre-trained Language Models enhanced Representation Learning for Conversational Recommender Systems [58.561904356651276]
  We introduce the Knowledge-Enhanced Entity Representation Learning (KERL) framework to improve the semantic understanding of entities for Conversational recommender systems. KERL uses a knowledge graph and a pre-trained language model to improve the semantic understanding of entities. KERL achieves state-of-the-art results in both recommendation and response generation tasks.
  arXiv  Detail & Related papers  (2023-12-18T06:41:23Z)
• Pre-training Language Models for Comparative Reasoning [26.161185103553635]
  We propose a novel framework to pre-train language models for enhancing their abilities of comparative reasoning over texts. Our approach introduces a novel method of collecting scalable data for text-based entity comparison. We present a framework of pre-training language models via three novel objectives on comparative reasoning.
  arXiv  Detail & Related papers  (2023-05-23T18:28:42Z)
• Beyond Contrastive Learning: A Variational Generative Model for Multilingual Retrieval [109.62363167257664]
  We propose a generative model for learning multilingual text embeddings. Our model operates on parallel data in $N$ languages. We evaluate this method on a suite of tasks including semantic similarity, bitext mining, and cross-lingual question retrieval.
  arXiv  Detail & Related papers  (2022-12-21T02:41:40Z)
• MetricBERT: Text Representation Learning via Self-Supervised Triplet Training [26.66640112616559]
  MetricBERT learns to embed text under a well-defined similarity metric. We show that MetricBERT outperforms state-of-the-art alternatives, sometimes by a substantial margin.
  arXiv  Detail & Related papers  (2022-08-13T09:52:58Z)
• Interpreting Language Models Through Knowledge Graph Extraction [42.97929497661778]
  We compare BERT-based language models through snapshots of acquired knowledge at sequential stages of the training process. We present a methodology to unveil a knowledge acquisition timeline by generating knowledge graph extracts from cloze "fill-in-the-blank" statements. We extend this analysis to a comparison of pretrained variations of BERT models (DistilBERT, BERT-base, RoBERTa)
  arXiv  Detail & Related papers  (2021-11-16T15:18:01Z)
• Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation [49.89831914386982]
  We propose a unified pre-trained language model (PLM) for all forms of text, including unstructured text, semi-structured text, and well-structured text. Our approach outperforms the pre-training of plain text using only 1/4 of the data.
  arXiv  Detail & Related papers  (2021-09-02T16:05:24Z)
• A Comprehensive Comparison of Pre-training Language Models [0.5139874302398955]
  We pre-train a list of transformer-based models with the same amount of text and the same training steps. The experimental results show that the most improvement upon the origin BERT is adding the RNN-layer to capture more contextual information for short text understanding.
  arXiv  Detail & Related papers  (2021-06-22T02:12:29Z)
• Context-Based Quotation Recommendation [60.93257124507105]
  We propose a novel context-aware quote recommendation system. It generates a ranked list of quotable paragraphs and spans of tokens from a given source document. We conduct experiments on a collection of speech transcripts and associated news articles.
  arXiv  Detail & Related papers  (2020-05-17T17:49:53Z)
• Coreferential Reasoning Learning for Language Representation [88.14248323659267]
  We present CorefBERT, a novel language representation model that can capture the coreferential relations in context. The experimental results show that, compared with existing baseline models, CorefBERT can achieve significant improvements consistently on various downstream NLP tasks.
  arXiv  Detail & Related papers  (2020-04-15T03:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.