Lightweight Self-Attentive Sequential Recommendation
- URL: http://arxiv.org/abs/2108.11333v1
- Date: Wed, 25 Aug 2021 16:46:47 GMT
- Title: Lightweight Self-Attentive Sequential Recommendation
- Authors: Yang Li, Tong Chen, Peng-Fei Zhang, Hongzhi Yin
- Abstract summary: We introduce a novel lightweight self-attentive network (LSAN) for sequential recommendation.
To aggressively compress the original embedding matrix, LSAN leverages the notion of compositional embeddings.
To account for the intrinsic dynamics of each item, we propose a temporal context-aware embedding composition scheme.
- Score: 30.048184102259494
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Modern deep neural networks (DNNs) have greatly facilitated the development
of sequential recommender systems by achieving state-of-the-art recommendation
performance on various sequential recommendation tasks. Given a sequence of
interacted items, existing DNN-based sequential recommenders commonly embed
each item into a unique vector to support subsequent computations of the user
interest. However, due to the potentially large number of items, the
over-parameterised item embedding matrix of a sequential recommender has become
a memory bottleneck for efficient deployment in resource-constrained
environments, e.g., smartphones and other edge devices. Furthermore, we observe
that the widely-used multi-head self-attention, though being effective in
modelling sequential dependencies among items, heavily relies on redundant
attention units to fully capture both global and local item-item transition
patterns within a sequence.
In this paper, we introduce a novel lightweight self-attentive network (LSAN)
for sequential recommendation. To aggressively compress the original embedding
matrix, LSAN leverages the notion of compositional embeddings, where each item
embedding is composed by merging a group of selected base embedding vectors
derived from substantially smaller embedding matrices. Meanwhile, to account
for the intrinsic dynamics of each item, we further propose a temporal
context-aware embedding composition scheme. Besides, we develop an innovative
twin-attention network that alleviates the redundancy of the traditional
multi-head self-attention while retaining full capacity for capturing long- and
short-term (i.e., global and local) item dependencies. Comprehensive
experiments demonstrate that LSAN significantly advances the accuracy and
memory efficiency of existing sequential recommenders.
Related papers
- CADMR: Cross-Attention and Disentangled Learning for Multimodal Recommender Systems [0.6037276428689637]
We propose CADMR, a novel autoencoder-based multimodal recommender system framework.
We evaluate CADMR on three benchmark datasets, demonstrating significant performance improvements over state-of-the-art methods.
arXiv Detail & Related papers (2024-12-03T09:09:52Z) - Unleash LLMs Potential for Recommendation by Coordinating Twin-Tower Dynamic Semantic Token Generator [60.07198935747619]
We propose Twin-Tower Dynamic Semantic Recommender (T TDS), the first generative RS which adopts dynamic semantic index paradigm.
To be more specific, we for the first time contrive a dynamic knowledge fusion framework which integrates a twin-tower semantic token generator into the LLM-based recommender.
The proposed T TDS recommender achieves an average improvement of 19.41% in Hit-Rate and 20.84% in NDCG metric, compared with the leading baseline methods.
arXiv Detail & Related papers (2024-09-14T01:45:04Z) - MISSRec: Pre-training and Transferring Multi-modal Interest-aware
Sequence Representation for Recommendation [61.45986275328629]
We propose MISSRec, a multi-modal pre-training and transfer learning framework for sequential recommendation.
On the user side, we design a Transformer-based encoder-decoder model, where the contextual encoder learns to capture the sequence-level multi-modal user interests.
On the candidate item side, we adopt a dynamic fusion module to produce user-adaptive item representation.
arXiv Detail & Related papers (2023-08-22T04:06:56Z) - Enhancing Few-shot NER with Prompt Ordering based Data Augmentation [59.69108119752584]
We propose a Prompt Ordering based Data Augmentation (PODA) method to improve the training of unified autoregressive generation frameworks.
Experimental results on three public NER datasets and further analyses demonstrate the effectiveness of our approach.
arXiv Detail & Related papers (2023-05-19T16:25:43Z) - Generative Slate Recommendation with Reinforcement Learning [49.75985313698214]
reinforcement learning algorithms can be used to optimize user engagement in recommender systems.
However, RL approaches are intractable in the slate recommendation scenario.
In that setting, an action corresponds to a slate that may contain any combination of items.
In this work we propose to encode slates in a continuous, low-dimensional latent space learned by a variational auto-encoder.
We are able to (i) relax assumptions required by previous work, and (ii) improve the quality of the action selection by modeling full slates.
arXiv Detail & Related papers (2023-01-20T15:28:09Z) - Multi-Behavior Hypergraph-Enhanced Transformer for Sequential
Recommendation [33.97708796846252]
We introduce a new Multi-Behavior Hypergraph-enhanced Transformer framework (MBHT) to capture both short-term and long-term cross-type behavior dependencies.
Specifically, a multi-scale Transformer is equipped with low-rank self-attention to jointly encode behavior-aware sequential patterns from fine-grained and coarse-grained levels.
arXiv Detail & Related papers (2022-07-12T15:07:21Z) - From STL Rulebooks to Rewards [4.859570041295978]
We propose a principled approach to shaping rewards for reinforcement learning from multiple objectives.
We first equip STL with a novel quantitative semantics allowing to automatically evaluate individual requirements.
We then develop a method for systematically combining evaluations of multiple requirements into a single reward.
arXiv Detail & Related papers (2021-10-06T14:16:59Z) - Sparse-Interest Network for Sequential Recommendation [78.83064567614656]
We propose a novel textbfSparse textbfInterest textbfNEtwork (SINE) for sequential recommendation.
Our sparse-interest module can adaptively infer a sparse set of concepts for each user from the large concept pool.
SINE can achieve substantial improvement over state-of-the-art methods.
arXiv Detail & Related papers (2021-02-18T11:03:48Z) - Sequential Recommendation with Self-Attentive Multi-Adversarial Network [101.25533520688654]
We present a Multi-Factor Generative Adversarial Network (MFGAN) for explicitly modeling the effect of context information on sequential recommendation.
Our framework is flexible to incorporate multiple kinds of factor information, and is able to trace how each factor contributes to the recommendation decision over time.
arXiv Detail & Related papers (2020-05-21T12:28:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.