Less is More: Information Bottleneck Denoised Multimedia Recommendation

• URL: http://arxiv.org/abs/2501.12175v1
• Date: Tue, 21 Jan 2025 14:33:07 GMT
• Title: Less is More: Information Bottleneck Denoised Multimedia Recommendation
• Authors: Yonghui Yang, Le Wu, Zhuangzhuang He, Zhengwei Wu, Richang Hong, Meng Wang,
• Abstract summary: We propose a denoised multimedia recommendation paradigm via the Information Bottleneck principle (IB)<n>IBMRec removes task-irrelevant features from both feature and item-item structure perspectives.<n>It is achieved by maximizing the mutual information between multimedia representation and recommendation tasks.
• Score: 43.66791467993419
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Empowered by semantic-rich content information, multimedia recommendation has emerged as a potent personalized technique. Current endeavors center around harnessing multimedia content to refine item representation or uncovering latent item-item structures based on modality similarity. Despite the effectiveness, we posit that these methods are usually suboptimal due to the introduction of irrelevant multimedia features into recommendation tasks. This stems from the fact that generic multimedia feature extractors, while well-designed for domain-specific tasks, can inadvertently introduce task-irrelevant features, leading to potential misguidance of recommenders. In this work, we propose a denoised multimedia recommendation paradigm via the Information Bottleneck principle (IB). Specifically, we propose a novel Information Bottleneck denoised Multimedia Recommendation (IBMRec) model to tackle the irrelevant feature issue. IBMRec removes task-irrelevant features from both feature and item-item structure perspectives, which are implemented by two-level IB learning modules: feature-level (FIB) and graph-level (GIB). In particular, FIB focuses on learning the minimal yet sufficient multimedia features. This is achieved by maximizing the mutual information between multimedia representation and recommendation tasks, while concurrently minimizing it between multimedia representation and pre-trained multimedia features. Furthermore, GIB is designed to learn the robust item-item graph structure, it refines the item-item graph based on preference affinity, then minimizes the mutual information between the original graph and the refined one. Extensive experiments across three benchmarks validate the effectiveness of our proposed model, showcasing high performance, and applicability to various multimedia recommenders.

Related papers

• Knowledge-aware Diffusion-Enhanced Multimedia Recommendation [9.12236232752614]
  We propose a Knowledge-aware Diffusion-Enhanced architecture using contrastive learning paradigms (KDiffE) for multimedia recommendations.<n>We first utilize original user-item graphs to build an attention-aware matrix into graph neural networks.<n>Then, we propose a guided diffusion model to generate strongly task-relevant knowledge graphs with less noise for constructing a knowledge-aware contrastive view.
  arXiv  Detail & Related papers  (2025-07-22T09:47:56Z)
• Towards Bridging the Cross-modal Semantic Gap for Multi-modal Recommendation [12.306686291299146]
  Multi-modal recommendation greatly enhances the performance of recommender systems. Most existing multi-modal recommendation models exploit multimedia information propagation processes to enrich item representations. We propose a novel framework to bridge the semantic gap between modalities and extract fine-grained multi-view semantic information.
  arXiv  Detail & Related papers  (2024-07-07T15:56:03Z)
• NoteLLM-2: Multimodal Large Representation Models for Recommendation [71.87790090964734]
  Large Language Models (LLMs) have demonstrated exceptional proficiency in text understanding and embedding tasks.<n>Their potential in multimodal representation, particularly for item-to-item (I2I) recommendations, remains underexplored.<n>We propose an end-to-end fine-tuning method that customizes the integration of any existing LLMs and vision encoders for efficient multimodal representation.
  arXiv  Detail & Related papers  (2024-05-27T03:24:01Z)
• Multimodal Pretraining and Generation for Recommendation: A Tutorial [54.07497722719509]
  The tutorial comprises three parts: multimodal pretraining, multimodal generation, and industrial applications. It aims to facilitate a swift understanding of multimodal recommendation and foster meaningful discussions on the future development of this evolving landscape.
  arXiv  Detail & Related papers  (2024-05-11T06:15:22Z)
• AlignRec: Aligning and Training in Multimodal Recommendations [29.995007279325947]
  multimodal recommendations can leverage rich contexts beyond interactions. Existing methods mainly regard multimodal information as an auxiliary, using them to help learn ID features. There exist semantic gaps among multimodal content features and ID-based features, for which directly using multimodal information as an auxiliary would lead to misalignment in representations of users and items.
  arXiv  Detail & Related papers  (2024-03-19T02:49:32Z)
• MONET: Modality-Embracing Graph Convolutional Network and Target-Aware Attention for Multimedia Recommendation [21.61057660080108]
  We focus on multimedia recommender systems using graph convolutional networks (GCNs) Our study aims to exploit multimodal features more effectively in order to accurately capture users' preferences for items. We propose a novel multimedia recommender system, named MONET, composed of following two core ideas: modality-embracing GCN (MeGCN) and target-aware attention.
  arXiv  Detail & Related papers  (2023-12-15T03:28:19Z)
• MM-GEF: Multi-modal representation meet collaborative filtering [43.88159639990081]
  We propose a graph-based item structure enhancement method MM-GEF: Multi-Modal recommendation with Graph Early-Fusion. MM-GEF learns refined item representations by injecting structural information obtained from both multi-modal and collaborative signals.
  arXiv  Detail & Related papers  (2023-08-14T15:47:36Z)
• Video-based Person Re-identification with Long Short-Term Representation Learning [101.62570747820541]
  Video-based person Re-Identification (V-ReID) aims to retrieve specific persons from raw videos captured by non-overlapped cameras. We propose a novel deep learning framework named Long Short-Term Representation Learning (LSTRL) for effective V-ReID.
  arXiv  Detail & Related papers  (2023-08-07T16:22:47Z)
• Latent Structures Mining with Contrastive Modality Fusion for Multimedia Recommendation [22.701371886522494]
  We argue that the latent semantic item-item structures underlying multimodal contents could be beneficial for learning better item representations. We devise a novel modality-aware structure learning module, which learns item-item relationships for each modality.
  arXiv  Detail & Related papers  (2021-11-01T03:37:02Z)
• Mining Latent Structures for Multimedia Recommendation [46.70109406399858]
  We propose a LATent sTructure mining method for multImodal reCommEndation, which we term LATTICE for brevity. We learn item-item structures for each modality and aggregates multiple modalities to obtain latent item graphs. Based on the learned latent graphs, we perform graph convolutions to explicitly inject high-order item affinities into item representations.
  arXiv  Detail & Related papers  (2021-04-19T03:50:24Z)
• VMSMO: Learning to Generate Multimodal Summary for Video-based News Articles [63.32111010686954]
  We propose the task of Video-based Multimodal Summarization with Multimodal Output (VMSMO) The main challenge in this task is to jointly model the temporal dependency of video with semantic meaning of article. We propose a Dual-Interaction-based Multimodal Summarizer (DIMS), consisting of a dual interaction module and multimodal generator.
  arXiv  Detail & Related papers  (2020-10-12T02:19:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.