Attention-based sequential recommendation system using multimodal data

• URL: http://arxiv.org/abs/2405.17959v1
• Date: Tue, 28 May 2024 08:41:05 GMT
• Title: Attention-based sequential recommendation system using multimodal data
• Authors: Hyungtaik Oh, Wonkeun Jo, Dongil Kim,
• Abstract summary: We propose an attention-based sequential recommendation method that employs multimodal data of items such as images, texts, and categories. The experimental results obtained from the Amazon datasets show that the proposed method outperforms those of conventional sequential recommendation systems.
• Score: 8.110978727364397
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sequential recommendation systems that model dynamic preferences based on a use's past behavior are crucial to e-commerce. Recent studies on these systems have considered various types of information such as images and texts. However, multimodal data have not yet been utilized directly to recommend products to users. In this study, we propose an attention-based sequential recommendation method that employs multimodal data of items such as images, texts, and categories. First, we extract image and text features from pre-trained VGG and BERT and convert categories into multi-labeled forms. Subsequently, attention operations are performed independent of the item sequence and multimodal representations. Finally, the individual attention information is integrated through an attention fusion function. In addition, we apply multitask learning loss for each modality to improve the generalization performance. The experimental results obtained from the Amazon datasets show that the proposed method outperforms those of conventional sequential recommendation systems.

Related papers

• Facet-Aware Multi-Head Mixture-of-Experts Model for Sequential Recommendation [25.516648802281626]
  We propose a novel structure called Facet-Aware Multi-Head Mixture-of-Experts Model for Sequential Recommendation (FAME) We leverage sub-embeddings from each head in the last multi-head attention layer to predict the next item separately. A gating mechanism integrates recommendations from each head and dynamically determines their importance.
  arXiv  Detail & Related papers  (2024-11-03T06:47:45Z)
• Ducho meets Elliot: Large-scale Benchmarks for Multimodal Recommendation [9.506245109666907]
  Multi-faceted features characterizing products and services may influence each customer on online selling platforms differently. The common multimodal recommendation pipeline involves (i) extracting multimodal features, (ii) refining their high-level representations to suit the recommendation task, and (iv) predicting the user-item score. This paper settles as the first attempt to offer a large-scale benchmarking for multimodal recommender systems, with a specific focus on multimodal extractors.
  arXiv  Detail & Related papers  (2024-09-24T08:29:10Z)
• Do We Really Need to Drop Items with Missing Modalities in Multimodal Recommendation? [15.428850539237182]
  We show that the lack of (some) modalities is, in fact, a widely-diffused phenomenon in multimodal recommendation. We propose a pipeline that imputes missing multimodal features in recommendation by leveraging traditional imputation strategies in machine learning.
  arXiv  Detail & Related papers  (2024-08-21T16:39:47Z)
• A Unified Graph Transformer for Overcoming Isolations in Multi-modal Recommendation [9.720586396359906]
  We argue that existing multi-modal recommender systems typically use isolated processes for both feature extraction and modality modelling. We propose a novel model, called Unified Multi-modal Graph Transformer (UGT), which leverages a multi-way transformer to extract aligned multi-modal features. We show that the UGT model can achieve significant effectiveness gains, especially when jointly optimised with the commonly-used multi-modal recommendation losses.
  arXiv  Detail & Related papers  (2024-07-29T11:04:31Z)
• MMGRec: Multimodal Generative Recommendation with Transformer Model [81.61896141495144]
  MMGRec aims to introduce a generative paradigm into multimodal recommendation. We first devise a hierarchical quantization method Graph CF-RQVAE to assign Rec-ID for each item from its multimodal information. We then train a Transformer-based recommender to generate the Rec-IDs of user-preferred items based on historical interaction sequences.
  arXiv  Detail & Related papers  (2024-04-25T12:11:27Z)
• BiVRec: Bidirectional View-based Multimodal Sequential Recommendation [55.87443627659778]
  We propose an innovative framework, BivRec, that jointly trains the recommendation tasks in both ID and multimodal views. BivRec achieves state-of-the-art performance on five datasets and showcases various practical advantages.
  arXiv  Detail & Related papers  (2024-02-27T09:10:41Z)
• Exploiting Modality-Specific Features For Multi-Modal Manipulation Detection And Grounding [54.49214267905562]
  We construct a transformer-based framework for multi-modal manipulation detection and grounding tasks. Our framework simultaneously explores modality-specific features while preserving the capability for multi-modal alignment. We propose an implicit manipulation query (IMQ) that adaptively aggregates global contextual cues within each modality.
  arXiv  Detail & Related papers  (2023-09-22T06:55:41Z)
• MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation [61.45986275328629]
  We propose MISSRec, a multi-modal pre-training and transfer learning framework for sequential recommendation. On the user side, we design a Transformer-based encoder-decoder model, where the contextual encoder learns to capture the sequence-level multi-modal user interests. On the candidate item side, we adopt a dynamic fusion module to produce user-adaptive item representation.
  arXiv  Detail & Related papers  (2023-08-22T04:06:56Z)
• Multi-Behavior Sequential Recommendation with Temporal Graph Transformer [66.10169268762014]
  We tackle the dynamic user-item relation learning with the awareness of multi-behavior interactive patterns. We propose a new Temporal Graph Transformer (TGT) recommendation framework to jointly capture dynamic short-term and long-range user-item interactive patterns.
  arXiv  Detail & Related papers  (2022-06-06T15:42:54Z)
• Knowledge-Enhanced Hierarchical Graph Transformer Network for Multi-Behavior Recommendation [56.12499090935242]
  This work proposes a Knowledge-Enhanced Hierarchical Graph Transformer Network (KHGT) to investigate multi-typed interactive patterns between users and items in recommender systems. KHGT is built upon a graph-structured neural architecture to capture type-specific behavior characteristics. We show that KHGT consistently outperforms many state-of-the-art recommendation methods across various evaluation settings.
  arXiv  Detail & Related papers  (2021-10-08T09:44:00Z)
• Large Scale Multimodal Classification Using an Ensemble of Transformer Models and Co-Attention [2.842794675894731]
  We describe our methodology and results for the SIGIR eCom Rakuten Data Challenge. We employ a dual attention technique to model image-text relationships using pretrained language and image embeddings.
  arXiv  Detail & Related papers  (2020-11-23T21:22:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.