A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarios

• URL: http://arxiv.org/abs/2409.17864v1
• Date: Thu, 26 Sep 2024 14:12:23 GMT
• Title: A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarios
• Authors: Christian Ganh\"or, Marta Moscati, Anna Hausberger, Shah Nawaz, Markus Schedl
• Abstract summary: We propose a novel technique for multimodal recommendation, relying on a multimodal Single-Branch embedding network for Recommendation (SiBraR) SiBraR encodes interaction data as well as multimodal side information using the same single-branch embedding network on different modalities. We show that SiBraR's recommendations are accurate in missing modality scenarios, and that the model is able to map different modalities to the same region of the shared embedding space.
• Score: 10.593268426317609
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most recommender systems adopt collaborative filtering (CF) and provide recommendations based on past collective interactions. Therefore, the performance of CF algorithms degrades when few or no interactions are available, a scenario referred to as cold-start. To address this issue, previous work relies on models leveraging both collaborative data and side information on the users or items. Similar to multimodal learning, these models aim at combining collaborative and content representations in a shared embedding space. In this work we propose a novel technique for multimodal recommendation, relying on a multimodal Single-Branch embedding network for Recommendation (SiBraR). Leveraging weight-sharing, SiBraR encodes interaction data as well as multimodal side information using the same single-branch embedding network on different modalities. This makes SiBraR effective in scenarios of missing modality, including cold start. Our extensive experiments on large-scale recommendation datasets from three different recommendation domains (music, movie, and e-commerce) and providing multimodal content information (audio, text, image, labels, and interactions) show that SiBraR significantly outperforms CF as well as state-of-the-art content-based RSs in cold-start scenarios, and is competitive in warm scenarios. We show that SiBraR's recommendations are accurate in missing modality scenarios, and that the model is able to map different modalities to the same region of the shared embedding space, hence reducing the modality gap.

Related papers

• FeDecider: An LLM-Based Framework for Federated Cross-Domain Recommendation [75.50721642765994]
  Large language model (LLM)-based recommendation models have demonstrated impressive performance.<n>We propose an LLM-based framework for Federated cross-domain recommendation, FeDecider.<n>Extensive experiments across diverse datasets validate the effectiveness of our proposed FeDecider.
  arXiv  Detail & Related papers  (2026-02-17T21:42:28Z)
• PRISM: Personalized Recommendation via Information Synergy Module [12.797662213207936]
  PRISM is a plug-and-play framework for sequential recommendation (SR)<n>It decomposes multimodal information into unique, redundant, and synergistic components.<n>Experiments on four datasets and three SR backbones demonstrate its effectiveness and versatility.
  arXiv  Detail & Related papers  (2026-01-16T02:17:54Z)
• Single-Branch Network Architectures to Close the Modality Gap in Multimodal Recommendation [10.048652639214275]
  We use single-branch neural networks equipped with weight sharing, modality sampling, and contrastive loss to provide accurate recommendations.<n>We compare these networks with multi-branch alternatives and conduct extensive experiments on three datasets.<n>Our results show that single-branch networks achieve competitive performance in warm-start scenarios and are significantly better in missing modality settings.
  arXiv  Detail & Related papers  (2025-09-23T08:58:53Z)
• Multi-modal Adaptive Mixture of Experts for Cold-start Recommendation [1.9967512860886603]
  MAMEX is a novel framework for multimodal cold-start recommendation.<n>It dynamically leverages latent representation from different modalities.<n>Experiments show MAMEX outperforms state-of-the-art methods in cold-start scenarios.
  arXiv  Detail & Related papers  (2025-08-11T14:47:14Z)
• Parameter-Efficient Single Collaborative Branch for Recommendation [11.572384740537016]
  We propose a novel RS that leverages weight sharing between the user and item NN modules used to obtain latent representations in the shared embedding space.<n>The proposed framework consists of a single Collaborative Branch for Recommendation (CoBraR)
  arXiv  Detail & Related papers  (2025-08-05T14:46:06Z)
• A Transfer Learning Framework for Multilayer Networks via Model Averaging [8.27209166988677]
  Link prediction in multilayer networks is a key challenge in applications such as recommendation systems and protein-protein interaction prediction.<n>We propose a novel transfer learning framework for multilayer networks using a bi-level model averaging method.
  arXiv  Detail & Related papers  (2025-06-14T11:32:31Z)
• Joint Similarity Item Exploration and Overlapped User Guidance for Multi-Modal Cross-Domain Recommendation [27.00142195880019]
  We propose Joint Similarity Item Exploration and Overlapped User Guidance (SIEOUG) for solving the Multi-Modal Cross-Domain Recommendation problem. Our empirical study on Amazon dataset with several different tasks demonstrates that SIEOUG significantly outperforms the state-of-the-art models under the MMCDR setting.
  arXiv  Detail & Related papers  (2025-02-22T03:57:43Z)
• Online Clustering of Dueling Bandits [59.09590979404303]
  We introduce the first "clustering of dueling bandit algorithms" to enable collaborative decision-making based on preference feedback. We propose two novel algorithms: (1) Clustering of Linear Dueling Bandits (COLDB) which models the user reward functions as linear functions of the context vectors, and (2) Clustering of Neural Dueling Bandits (CONDB) which uses a neural network to model complex, non-linear user reward functions.
  arXiv  Detail & Related papers  (2025-02-04T07:55:41Z)
• Offline Learning for Combinatorial Multi-armed Bandits [56.96242764723241]
  Off-CMAB is the first offline learning framework for CMAB. Off-CMAB combines pessimistic reward estimations with solvers. Experiments on synthetic and real-world datasets highlight the superior performance of CLCB.
  arXiv  Detail & Related papers  (2025-01-31T16:56:18Z)
• Towards Bridging the Cross-modal Semantic Gap for Multi-modal Recommendation [12.306686291299146]
  Multi-modal recommendation greatly enhances the performance of recommender systems. Most existing multi-modal recommendation models exploit multimedia information propagation processes to enrich item representations. We propose a novel framework to bridge the semantic gap between modalities and extract fine-grained multi-view semantic information.
  arXiv  Detail & Related papers  (2024-07-07T15:56:03Z)
• Combinatorial Multivariant Multi-Armed Bandits with Applications to Episodic Reinforcement Learning and Beyond [58.39457881271146]
  We introduce a novel framework of multi-armed bandits (CMAB) with multivariant and probabilistically triggering arms (CMAB-MT) Compared with existing CMAB works, CMAB-MT not only enhances the modeling power but also allows improved results by leveraging distinct statistical properties for multivariant random variables. Our framework can include many important problems as applications, such as episodic reinforcement learning (RL) and probabilistic maximum coverage for goods distribution.
  arXiv  Detail & Related papers  (2024-06-03T14:48:53Z)
• RecFusion: A Binomial Diffusion Process for 1D Data for Recommendation [48.77168472848952]
  We propose RecFusion, which comprises a set of diffusion models for recommendation. We formulate diffusion on a 1D vector and propose binomial diffusion, which explicitly models binary user-item interactions with a Bernoulli process. Our proposed diffusion models have implications beyond recommendation systems, such as in the medical domain with MRI and CT scans.
  arXiv  Detail & Related papers  (2023-06-15T08:39:24Z)
• Diverse Preference Augmentation with Multiple Domains for Cold-start Recommendations [92.47380209981348]
  We propose a Diverse Preference Augmentation framework with multiple source domains based on meta-learning. We generate diverse ratings in a new domain of interest to handle overfitting on the case of sparse interactions. These ratings are introduced into the meta-training procedure to learn a preference meta-learner, which produces good generalization ability.
  arXiv  Detail & Related papers  (2022-04-01T10:10:50Z)
• DSKReG: Differentiable Sampling on Knowledge Graph for Recommendation with Relational GNN [59.160401038969795]
  We propose differentiable sampling on Knowledge Graph for Recommendation with GNN (DSKReG) We devise a differentiable sampling strategy, which enables the selection of relevant items to be jointly optimized with the model training procedure. The experimental results demonstrate that our model outperforms state-of-the-art KG-based recommender systems.
  arXiv  Detail & Related papers  (2021-08-26T16:19:59Z)
• Thematic recommendations on knowledge graphs using multilayer networks [0.0]
  We present a framework to generate and evaluate thematic recommendations based on multilayer network representations of knowledge graphs (KGs) In this representation, each layer encodes a different type of relationship in the KG, and directed interlayer couplings connect the same entity in different roles. We apply an adaptation of the personalised PageRank algorithm to multilayer models of KGs to generate item-item recommendations.
  arXiv  Detail & Related papers  (2021-05-12T15:30:21Z)
• Scenario-aware and Mutual-based approach for Multi-scenario Recommendation in E-Commerce [12.794276204716642]
  How to make accurate recommendations for users in heterogeneous e-commerce scenarios is still a continuous research topic. We propose a novel recommendation model named Scenario-aware Mutual Learning (SAML) that leverages the differences and similarities between multiple scenarios.
  arXiv  Detail & Related papers  (2020-12-16T13:52:14Z)
• Deep Multimodal Fusion by Channel Exchanging [87.40768169300898]
  This paper proposes a parameter-free multimodal fusion framework that dynamically exchanges channels between sub-networks of different modalities. The validity of such exchanging process is also guaranteed by sharing convolutional filters yet keeping separate BN layers across modalities, which, as an add-on benefit, allows our multimodal architecture to be almost as compact as a unimodal network.
  arXiv  Detail & Related papers  (2020-11-10T09:53:20Z)
• An efficient manifold density estimator for all recommendation systems [3.2981402185055213]
  We propose a framework utilizing arbitrary vector representations with the property of local similarity to succinctly represent smooth probability densities. Our approximate representation has the desirable properties of being fixed-size and having simple additive compositionality, thus being especially amenable to treatment with neural networks. Applying E to both top-k and session-based recommendation settings, we establish new state-of-the-art results on multiple open datasets in both uni-modal and multi-modal settings.
  arXiv  Detail & Related papers  (2020-06-02T19:20:20Z)
• Seamlessly Unifying Attributes and Items: Conversational Recommendation for Cold-Start Users [111.28351584726092]
  We consider the conversational recommendation for cold-start users, where a system can both ask the attributes from and recommend items to a user interactively. Our Conversational Thompson Sampling (ConTS) model holistically solves all questions in conversational recommendation by choosing the arm with the maximal reward to play.
  arXiv  Detail & Related papers  (2020-05-23T08:56:37Z)
• COBRA: Contrastive Bi-Modal Representation Algorithm [43.33840912256077]
  We present a novel framework that aims to train two modalities in a joint fashion inspired by Contrastive Predictive Coding (CPC) and Noise Contrastive Estimation (NCE) paradigms. We empirically show that this framework reduces the modality gap significantly and generates a robust and task agnostic joint-embedding space. We outperform existing work on four diverse downstream tasks spanning across seven benchmark cross-modal datasets.
  arXiv  Detail & Related papers  (2020-05-07T18:20:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.