Multi-Margin Cosine Loss: Proposal and Application in Recommender Systems

• URL: http://arxiv.org/abs/2405.04614v3
• Date: Tue, 10 Sep 2024 14:37:00 GMT
• Title: Multi-Margin Cosine Loss: Proposal and Application in Recommender Systems
• Authors: Makbule Gulcin Ozsoy,
• Abstract summary: Collaborative filtering-based deep learning techniques have regained popularity due to their straightforward nature. These systems consist of three main components: an interaction module, a loss function, and a negative sampling strategy. The proposed Multi-Margin Cosine Loss (MMCL) addresses these challenges by introducing multiple margins and varying weights for negative samples.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Recommender systems guide users through vast amounts of information by suggesting items based on their predicted preferences. Collaborative filtering-based deep learning techniques have regained popularity due to their straightforward nature, relying only on user-item interactions. Typically, these systems consist of three main components: an interaction module, a loss function, and a negative sampling strategy. Initially, researchers focused on enhancing performance by developing complex interaction modules. However, there has been a recent shift toward refining loss functions and negative sampling strategies. This shift has led to an increased interest in contrastive learning, which pulls similar pairs closer while pushing dissimilar ones apart. Contrastive learning may bring challenges like high memory demands and under-utilization of some negative samples. The proposed Multi-Margin Cosine Loss (MMCL) addresses these challenges by introducing multiple margins and varying weights for negative samples. It efficiently utilizes not only the hardest negatives but also other non-trivial negatives, offers a simpler yet effective loss function that outperforms more complex methods, especially when resources are limited. Experiments on two well-known datasets demonstrated that MMCL achieved up to a 20\% performance improvement compared to a baseline loss function when fewer number of negative samples are used.

Related papers

• SimCE: Simplifying Cross-Entropy Loss for Collaborative Filtering [47.81610130269399]
  We propose a Sampled Softmax Cross-Entropy (SSM) that compares one positive sample with multiple negative samples, leading to better performance. We also introduce a underlineSimplified Sampled Softmax underlineCross-underlineEntropy Loss (SimCE) which simplifies the SSM using its upper bound. Our validation on 12 benchmark datasets, using both MF and LightGCN backbones, shows that SimCE significantly outperforms both BPR and SSM.
  arXiv  Detail & Related papers  (2024-06-23T17:24:07Z)
• Rethinking Negative Pairs in Code Search [56.23857828689406]
  We propose a simple yet effective Soft-InfoNCE loss that inserts weight terms into InfoNCE. We analyze the effects of Soft-InfoNCE on controlling the distribution of learnt code representations and on deducing a more precise mutual information estimation.
  arXiv  Detail & Related papers  (2023-10-12T06:32:42Z)
• Generating Negative Samples for Sequential Recommendation [83.60655196391855]
  We propose to Generate Negative Samples (items) for Sequential Recommendation (SR) A negative item is sampled at each time step based on the current SR model's learned user preferences toward items. Experiments on four public datasets verify the importance of providing high-quality negative samples for SR.
  arXiv  Detail & Related papers  (2022-08-07T05:44:13Z)
• C$^{4}$Net: Contextual Compression and Complementary Combination Network for Salient Object Detection [0.0]
  We show that feature concatenation works better than other combination methods like multiplication or addition. Also, joint feature learning gives better results, because of the information sharing during their processing.
  arXiv  Detail & Related papers  (2021-10-22T16:14:10Z)
• Multi-Sample based Contrastive Loss for Top-k Recommendation [33.02297142668278]
  The Contrastive Loss (CL) is the key in contrastive learning that has received more attention recently. We propose a new data augmentation method by using multiple positive items (or samples) simultaneously with the CL loss function.
  arXiv  Detail & Related papers  (2021-09-01T07:32:13Z)
• LoOp: Looking for Optimal Hard Negative Embeddings for Deep Metric Learning [17.571160136568455]
  We propose a novel approach that looks for optimal hard negatives (LoOp) in the embedding space. Unlike mining-based methods, our approach considers the entire space between pairs of embeddings to calculate the optimal hard negatives.
  arXiv  Detail & Related papers  (2021-08-20T19:21:33Z)
• Rethinking Deep Contrastive Learning with Embedding Memory [58.66613563148031]
  Pair-wise loss functions have been extensively studied and shown to continuously improve the performance of deep metric learning (DML) We provide a new methodology for systematically studying weighting strategies of various pair-wise loss functions, and rethink pair weighting with an embedding memory.
  arXiv  Detail & Related papers  (2021-03-25T17:39:34Z)
• Contrastive Learning with Hard Negative Samples [80.12117639845678]
  We develop a new family of unsupervised sampling methods for selecting hard negative samples. A limiting case of this sampling results in a representation that tightly clusters each class, and pushes different classes as far apart as possible. The proposed method improves downstream performance across multiple modalities, requires only few additional lines of code to implement, and introduces no computational overhead.
  arXiv  Detail & Related papers  (2020-10-09T14:18:53Z)
• Multi-scale Interactive Network for Salient Object Detection [91.43066633305662]
  We propose the aggregate interaction modules to integrate the features from adjacent levels. To obtain more efficient multi-scale features, the self-interaction modules are embedded in each decoder unit. Experimental results on five benchmark datasets demonstrate that the proposed method without any post-processing performs favorably against 23 state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-07-17T15:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.