SimCE: Simplifying Cross-Entropy Loss for Collaborative Filtering

• URL: http://arxiv.org/abs/2406.16170v1
• Date: Sun, 23 Jun 2024 17:24:07 GMT
• Title: SimCE: Simplifying Cross-Entropy Loss for Collaborative Filtering
• Authors: Xiaodong Yang, Huiyuan Chen, Yuchen Yan, Yuxin Tang, Yuying Zhao, Eric Xu, Yiwei Cai, Hanghang Tong,
• Abstract summary: We propose a Sampled Softmax Cross-Entropy (SSM) that compares one positive sample with multiple negative samples, leading to better performance. We also introduce a underlineSimplified Sampled Softmax underlineCross-underlineEntropy Loss (SimCE) which simplifies the SSM using its upper bound. Our validation on 12 benchmark datasets, using both MF and LightGCN backbones, shows that SimCE significantly outperforms both BPR and SSM.
• Score: 47.81610130269399
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The learning objective is integral to collaborative filtering systems, where the Bayesian Personalized Ranking (BPR) loss is widely used for learning informative backbones. However, BPR often experiences slow convergence and suboptimal local optima, partially because it only considers one negative item for each positive item, neglecting the potential impacts of other unobserved items. To address this issue, the recently proposed Sampled Softmax Cross-Entropy (SSM) compares one positive sample with multiple negative samples, leading to better performance. Our comprehensive experiments confirm that recommender systems consistently benefit from multiple negative samples during training. Furthermore, we introduce a \underline{Sim}plified Sampled Softmax \underline{C}ross-\underline{E}ntropy Loss (SimCE), which simplifies the SSM using its upper bound. Our validation on 12 benchmark datasets, using both MF and LightGCN backbones, shows that SimCE significantly outperforms both BPR and SSM.

Related papers

• Multi-Margin Cosine Loss: Proposal and Application in Recommender Systems [0.0]
  Collaborative filtering-based deep learning techniques have regained popularity due to their straightforward nature. These systems consist of three main components: an interaction module, a loss function, and a negative sampling strategy. The proposed Multi-Margin Cosine Loss (MMCL) addresses these challenges by introducing multiple margins and varying weights for negative samples.
  arXiv  Detail & Related papers  (2024-05-07T18:58:32Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• On the Theories Behind Hard Negative Sampling for Recommendation [51.64626293229085]
  We offer two insightful guidelines for effective usage of Hard Negative Sampling (HNS) We prove that employing HNS on the Personalized Ranking (BPR) learner is equivalent to optimizing One-way Partial AUC (OPAUC) These analyses establish the theoretical foundation of HNS in optimizing Top-K recommendation performance for the first time.
  arXiv  Detail & Related papers  (2023-02-07T13:57:03Z)
• Rethinking Collaborative Metric Learning: Toward an Efficient Alternative without Negative Sampling [156.7248383178991]
  Collaborative Metric Learning (CML) paradigm has aroused wide interest in the area of recommendation systems (RS) We find that negative sampling would lead to a biased estimation of the generalization error. Motivated by this, we propose an efficient alternative without negative sampling for CML named textitSampling-Free Collaborative Metric Learning (SFCML)
  arXiv  Detail & Related papers  (2022-06-23T08:50:22Z)
• Hard Negative Sampling via Regularized Optimal Transport for Contrastive Representation Learning [13.474603286270836]
  We study the problem of designing hard negative sampling distributions for unsupervised contrastive representation learning. We propose and analyze a novel min-max framework that seeks a representation which minimizes the maximum (worst-case) generalized contrastive learning loss.
  arXiv  Detail & Related papers  (2021-11-04T21:25:24Z)
• Multi-Sample based Contrastive Loss for Top-k Recommendation [33.02297142668278]
  The Contrastive Loss (CL) is the key in contrastive learning that has received more attention recently. We propose a new data augmentation method by using multiple positive items (or samples) simultaneously with the CL loss function.
  arXiv  Detail & Related papers  (2021-09-01T07:32:13Z)
• Reenvisioning Collaborative Filtering vs Matrix Factorization [65.74881520196762]
  Collaborative filtering models based on matrix factorization and learned similarities using Artificial Neural Networks (ANNs) have gained significant attention in recent years. Announcement of ANNs within the recommendation ecosystem has been recently questioned, raising several comparisons in terms of efficiency and effectiveness. We show the potential these techniques may have on beyond-accuracy evaluation while analyzing effect on complementary evaluation dimensions.
  arXiv  Detail & Related papers  (2021-07-28T16:29:38Z)
• Contrastive Attraction and Contrastive Repulsion for Representation Learning [131.72147978462348]
  Contrastive learning (CL) methods learn data representations in a self-supervision manner, where the encoder contrasts each positive sample over multiple negative samples. Recent CL methods have achieved promising results when pretrained on large-scale datasets, such as ImageNet. We propose a doubly CL strategy that separately compares positive and negative samples within their own groups, and then proceeds with a contrast between positive and negative groups.
  arXiv  Detail & Related papers  (2021-05-08T17:25:08Z)
• M$^5$L: Multi-Modal Multi-Margin Metric Learning for RGBT Tracking [44.296318907168]
  Classifying the confusing samples in the course of RGBT tracking is a challenging problem. We propose a novel Multi-Modal Multi-Margin Metric Learning framework, named M$5$L for RGBT tracking. Our framework clearly improves the tracking performance and outperforms the state-of-the-art RGBT trackers.
  arXiv  Detail & Related papers  (2020-03-17T11:37:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.