Learning Self-Modulating Attention in Continuous Time Space with Applications to Sequential Recommendation

• URL: http://arxiv.org/abs/2204.06517v1
• Date: Wed, 30 Mar 2022 03:54:11 GMT
• Title: Learning Self-Modulating Attention in Continuous Time Space with Applications to Sequential Recommendation
• Authors: Chao Chen, Haoyu Geng, Nianzu Yang, Junchi Yan, Daiyue Xue, Jianping Yu and Xiaokang Yang
• Abstract summary: We propose a novel attention network, named self-modulating attention, that models the complex and non-linearly evolving dynamic user preferences. We empirically demonstrate the effectiveness of our method on top-N sequential recommendation tasks, and the results on three large-scale real-world datasets show that our model can achieve state-of-the-art performance.
• Score: 102.24108167002252
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: User interests are usually dynamic in the real world, which poses both theoretical and practical challenges for learning accurate preferences from rich behavior data. Among existing user behavior modeling solutions, attention networks are widely adopted for its effectiveness and relative simplicity. Despite being extensively studied, existing attentions still suffer from two limitations: i) conventional attentions mainly take into account the spatial correlation between user behaviors, regardless the distance between those behaviors in the continuous time space; and ii) these attentions mostly provide a dense and undistinguished distribution over all past behaviors then attentively encode them into the output latent representations. This is however not suitable in practical scenarios where a user's future actions are relevant to a small subset of her/his historical behaviors. In this paper, we propose a novel attention network, named self-modulating attention, that models the complex and non-linearly evolving dynamic user preferences. We empirically demonstrate the effectiveness of our method on top-N sequential recommendation tasks, and the results on three large-scale real-world datasets show that our model can achieve state-of-the-art performance.

Related papers

• Long-Sequence Recommendation Models Need Decoupled Embeddings [49.410906935283585]
  We identify and characterize a neglected deficiency in existing long-sequence recommendation models. A single set of embeddings struggles with learning both attention and representation, leading to interference between these two processes. We propose the Decoupled Attention and Representation Embeddings (DARE) model, where two distinct embedding tables are learned separately to fully decouple attention and representation.
  arXiv  Detail & Related papers  (2024-10-03T15:45:15Z)
• Personalized Behavior-Aware Transformer for Multi-Behavior Sequential Recommendation [25.400756652696895]
  We propose a Personalized Behavior-Aware Transformer framework (PBAT) for Multi-Behavior Sequential Recommendation (MBSR) problem. PBAT develops a personalized behavior pattern generator in the representation layer, which extracts dynamic and discriminative behavior patterns for sequential learning. We conduct experiments on three benchmark datasets and the results demonstrate the effectiveness and interpretability of our framework.
  arXiv  Detail & Related papers  (2024-02-22T12:03:21Z)
• Learning Dual Dynamic Representations on Time-Sliced User-Item Interaction Graphs for Sequential Recommendation [62.30552176649873]
  We devise a novel Dynamic Representation Learning model for Sequential Recommendation (DRL-SRe) To better model the user-item interactions for characterizing the dynamics from both sides, the proposed model builds a global user-item interaction graph for each time slice. To enable the model to capture fine-grained temporal information, we propose an auxiliary temporal prediction task over consecutive time slices.
  arXiv  Detail & Related papers  (2021-09-24T07:44:27Z)
• Dynamic Memory based Attention Network for Sequential Recommendation [79.5901228623551]
  We propose a novel long sequential recommendation model called Dynamic Memory-based Attention Network (DMAN) It segments the overall long behavior sequence into a series of sub-sequences, then trains the model and maintains a set of memory blocks to preserve long-term interests of users. Based on the dynamic memory, the user's short-term and long-term interests can be explicitly extracted and combined for efficient joint recommendation.
  arXiv  Detail & Related papers  (2021-02-18T11:08:54Z)
• Multi-Interactive Attention Network for Fine-grained Feature Learning in CTR Prediction [48.267995749975476]
  In the Click-Through Rate (CTR) prediction scenario, user's sequential behaviors are well utilized to capture the user interest. Existing methods mostly utilize attention on the behavior of users, which is not always suitable for CTR prediction. We propose a Multi-Interactive Attention Network (MIAN) to comprehensively extract the latent relationship among all kinds of fine-grained features.
  arXiv  Detail & Related papers  (2020-12-13T05:46:19Z)
• Cost-effective Interactive Attention Learning with Neural Attention Processes [79.8115563067513]
  We propose a novel interactive learning framework which we refer to as Interactive Attention Learning (IAL) IAL is prone to overfitting due to scarcity of human annotations, and requires costly retraining. We tackle these challenges by proposing a sample-efficient attention mechanism and a cost-effective reranking algorithm for instances and features.
  arXiv  Detail & Related papers  (2020-06-09T17:36:41Z)
• A Spatial-Temporal Attentive Network with Spatial Continuity for Trajectory Prediction [74.00750936752418]
  We propose a novel model named spatial-temporal attentive network with spatial continuity (STAN-SC) First, spatial-temporal attention mechanism is presented to explore the most useful and important information. Second, we conduct a joint feature sequence based on the sequence and instant state information to make the generative trajectories keep spatial continuity.
  arXiv  Detail & Related papers  (2020-03-13T04:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.