Single-Branch Network Architectures to Close the Modality Gap in Multimodal Recommendation

• URL: http://arxiv.org/abs/2509.18807v1
• Date: Tue, 23 Sep 2025 08:58:53 GMT
• Title: Single-Branch Network Architectures to Close the Modality Gap in Multimodal Recommendation
• Authors: Christian Ganhör, Marta Moscati, Anna Hausberger, Shah Nawaz, Markus Schedl,
• Abstract summary: We use single-branch neural networks equipped with weight sharing, modality sampling, and contrastive loss to provide accurate recommendations.<n>We compare these networks with multi-branch alternatives and conduct extensive experiments on three datasets.<n>Our results show that single-branch networks achieve competitive performance in warm-start scenarios and are significantly better in missing modality settings.
• Score: 10.048652639214275
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional recommender systems rely on collaborative filtering, using past user-item interactions to help users discover new items in a vast collection. In cold start, i.e., when interaction histories of users or items are not available, content-based recommender systems use side information instead. Hybrid recommender systems (HRSs) often employ multimodal learning to combine collaborative and side information, which we jointly refer to as modalities. Though HRSs can provide recommendations when some modalities are missing, their quality degrades. In this work, we utilize single-branch neural networks equipped with weight sharing, modality sampling, and contrastive loss to provide accurate recommendations even in missing modality scenarios by narrowing the modality gap. We compare these networks with multi-branch alternatives and conduct extensive experiments on three datasets. Six accuracy-based and four beyond-accuracy-based metrics help assess the recommendation quality for the different training paradigms and their hyperparameters in warm-start and missing modality scenarios. We quantitatively and qualitatively study the effects of these different aspects on bridging the modality gap. Our results show that single-branch networks achieve competitive performance in warm-start scenarios and are significantly better in missing modality settings. Moreover, our approach leads to closer proximity of an item's modalities in the embedding space. Our full experimental setup is available at https://github.com/hcai-mms/single-branch-networks.

Related papers

• Collaborative Group-Aware Hashing for Fast Recommender Systems [66.92426381995695]
  Hash technique has shown its superiority for speeding up the online recommendation by bit operations on Hamming distance computations.<n>Existing hashing-based recommendations suffer from low accuracy, especially with sparse settings.<n>This paper lodges a Collaborative Group-Aware Hashing (CGAH) method for both collaborative filtering and content-aware recommendations.
  arXiv  Detail & Related papers  (2025-12-23T09:07:28Z)
• Multi-modal Adaptive Mixture of Experts for Cold-start Recommendation [1.9967512860886603]
  MAMEX is a novel framework for multimodal cold-start recommendation.<n>It dynamically leverages latent representation from different modalities.<n>Experiments show MAMEX outperforms state-of-the-art methods in cold-start scenarios.
  arXiv  Detail & Related papers  (2025-08-11T14:47:14Z)
• Why am I seeing this? Towards recognizing social media recommender systems with missing recommendations [4.242821809663174]
  We introduce a method for Automatic Recommender Recognition using Graph Neural Networks (GNNs)<n>Our approach enables accurate detection of hidden recommenders and their influence on user behavior.<n>This study provides insights into how recommenders shape behavior, aiding efforts to reduce polarization and misinformation.
  arXiv  Detail & Related papers  (2025-04-15T09:16:17Z)
• Online Clustering of Dueling Bandits [59.09590979404303]
  We introduce the first "clustering of dueling bandit algorithms" to enable collaborative decision-making based on preference feedback.<n>We propose two novel algorithms: (1) Clustering of Linear Dueling Bandits (COLDB) which models the user reward functions as linear functions of the context vectors, and (2) Clustering of Neural Dueling Bandits (CONDB) which uses a neural network to model complex, non-linear user reward functions.
  arXiv  Detail & Related papers  (2025-02-04T07:55:41Z)
• A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarios [10.593268426317609]
  We propose a novel technique for multimodal recommendation, relying on a multimodal Single-Branch embedding network for Recommendation (SiBraR) SiBraR encodes interaction data as well as multimodal side information using the same single-branch embedding network on different modalities. We show that SiBraR's recommendations are accurate in missing modality scenarios, and that the model is able to map different modalities to the same region of the shared embedding space.
  arXiv  Detail & Related papers  (2024-09-26T14:12:23Z)
• Mirror Gradient: Towards Robust Multimodal Recommender Systems via Exploring Flat Local Minima [54.06000767038741]
  We analyze multimodal recommender systems from the novel perspective of flat local minima. We propose a concise yet effective gradient strategy called Mirror Gradient (MG) We find that the proposed MG can complement existing robust training methods and be easily extended to diverse advanced recommendation models.
  arXiv  Detail & Related papers  (2024-02-17T12:27:30Z)
• Broad Recommender System: An Efficient Nonlinear Collaborative Filtering Approach [56.12815715932561]
  We propose a new broad recommender system called Broad Collaborative Filtering (BroadCF) Instead of Deep Neural Networks (DNNs), Broad Learning System (BLS) is used as a mapping function to learn the complex nonlinear relationships between users and items. Extensive experiments conducted on seven benchmark datasets have confirmed the effectiveness of the proposed BroadCF algorithm.
  arXiv  Detail & Related papers  (2022-04-20T01:25:08Z)
• Diverse Preference Augmentation with Multiple Domains for Cold-start Recommendations [92.47380209981348]
  We propose a Diverse Preference Augmentation framework with multiple source domains based on meta-learning. We generate diverse ratings in a new domain of interest to handle overfitting on the case of sparse interactions. These ratings are introduced into the meta-training procedure to learn a preference meta-learner, which produces good generalization ability.
  arXiv  Detail & Related papers  (2022-04-01T10:10:50Z)
• Collaborative Reflection-Augmented Autoencoder Network for Recommender Systems [23.480069921831344]
  We develop a Collaborative Reflection-Augmented Autoencoder Network (CRANet) CRANet is capable of exploring transferable knowledge from observed and unobserved user-item interactions. We experimentally validate CRANet on four diverse benchmark datasets corresponding to two recommendation tasks.
  arXiv  Detail & Related papers  (2022-01-10T04:36:15Z)
• Hyper Meta-Path Contrastive Learning for Multi-Behavior Recommendation [61.114580368455236]
  User purchasing prediction with multi-behavior information remains a challenging problem for current recommendation systems. We propose the concept of hyper meta-path to construct hyper meta-paths or hyper meta-graphs to explicitly illustrate the dependencies among different behaviors of a user. Thanks to the recent success of graph contrastive learning, we leverage it to learn embeddings of user behavior patterns adaptively instead of assigning a fixed scheme to understand the dependencies among different behaviors.
  arXiv  Detail & Related papers  (2021-09-07T04:28:09Z)
• DSKReG: Differentiable Sampling on Knowledge Graph for Recommendation with Relational GNN [59.160401038969795]
  We propose differentiable sampling on Knowledge Graph for Recommendation with GNN (DSKReG) We devise a differentiable sampling strategy, which enables the selection of relevant items to be jointly optimized with the model training procedure. The experimental results demonstrate that our model outperforms state-of-the-art KG-based recommender systems.
  arXiv  Detail & Related papers  (2021-08-26T16:19:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.