Modeling Sequences as Distributions with Uncertainty for Sequential
Recommendation
- URL: http://arxiv.org/abs/2106.06165v1
- Date: Fri, 11 Jun 2021 04:35:21 GMT
- Title: Modeling Sequences as Distributions with Uncertainty for Sequential
Recommendation
- Authors: Ziwei Fan, Zhiwei Liu, Lei Zheng, Shen Wang, Philip S. Yu
- Abstract summary: Most existing sequential methods assume users are deterministic.
Item-item transitions might fluctuate significantly in several item aspects and exhibit randomness of user interests.
We propose a Distribution-based Transformer Sequential Recommendation (DT4SR) which injects uncertainties into sequential modeling.
- Score: 63.77513071533095
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: The sequential patterns within the user interactions are pivotal for
representing the user's preference and capturing latent relationships among
items. The recent advancements of sequence modeling by Transformers advocate
the community to devise more effective encoders for the sequential
recommendation. Most existing sequential methods assume users are
deterministic. However, item-item transitions might fluctuate significantly in
several item aspects and exhibit randomness of user interests. This
\textit{stochastic characteristics} brings up a solid demand to include
uncertainties in representing sequences and items. Additionally, modeling
sequences and items with uncertainties expands users' and items' interaction
spaces, thus further alleviating cold-start problems.
In this work, we propose a Distribution-based Transformer for Sequential
Recommendation (DT4SR), which injects uncertainties into sequential modeling.
We use Elliptical Gaussian distributions to describe items and sequences with
uncertainty. We describe the uncertainty in items and sequences as Elliptical
Gaussian distribution. And we adopt Wasserstein distance to measure the
similarity between distributions. We devise two novel Trans-formers for
modeling mean and covariance, which guarantees the positive-definite property
of distributions. The proposed method significantly outperforms the
state-of-the-art methods. The experiments on three benchmark datasets also
demonstrate its effectiveness in alleviating cold-start issues. The code is
available inhttps://github.com/DyGRec/DT4SR.
Related papers
- Mutual Exclusivity Training and Primitive Augmentation to Induce
Compositionality [84.94877848357896]
Recent datasets expose the lack of the systematic generalization ability in standard sequence-to-sequence models.
We analyze this behavior of seq2seq models and identify two contributing factors: a lack of mutual exclusivity bias and the tendency to memorize whole examples.
We show substantial empirical improvements using standard sequence-to-sequence models on two widely-used compositionality datasets.
arXiv Detail & Related papers (2022-11-28T17:36:41Z) - Sequential Recommendation via Stochastic Self-Attention [68.52192964559829]
Transformer-based approaches embed items as vectors and use dot-product self-attention to measure the relationship between items.
We propose a novel textbfSTOchastic textbfSelf-textbfAttention(STOSA) to overcome these issues.
We devise a novel Wasserstein Self-Attention module to characterize item-item position-wise relationships in sequences.
arXiv Detail & Related papers (2022-01-16T12:38:45Z) - Two-level monotonic multistage recommender systems [5.983189537988243]
Two-level monotonic property characterizing a monotonic chain of events for personalized prediction.
Regularized cost function to learn user-specific behaviors at different stages.
Algorithm based on blockwise coordinate descent.
arXiv Detail & Related papers (2021-10-06T08:50:32Z) - DeepExpress: Heterogeneous and Coupled Sequence Modeling for Express
Delivery Prediction [41.49535437719119]
A precise estimate of consumer delivery requests has to involve sequential factors such as shopping behaviors, weather conditions, events, business campaigns, and their couplings.
DeepExpress is a deep-learning based express delivery sequence prediction model, which extends the classic seq2seq framework to learning complex coupling between sequence and features.
arXiv Detail & Related papers (2021-08-18T14:24:19Z) - Contrastive Self-supervised Sequential Recommendation with Robust
Augmentation [101.25762166231904]
Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data.
Old and new issues remain, including data-sparsity and noisy data.
We propose Contrastive Self-Supervised Learning for sequential Recommendation (CoSeRec)
arXiv Detail & Related papers (2021-08-14T07:15:25Z) - Multivariate Probabilistic Regression with Natural Gradient Boosting [63.58097881421937]
We propose a Natural Gradient Boosting (NGBoost) approach based on nonparametrically modeling the conditional parameters of the multivariate predictive distribution.
Our method is robust, works out-of-the-box without extensive tuning, is modular with respect to the assumed target distribution, and performs competitively in comparison to existing approaches.
arXiv Detail & Related papers (2021-06-07T17:44:49Z) - Variation Control and Evaluation for Generative SlateRecommendations [22.533997063750597]
We show that item perturbation can enforce slate variation and mitigate the over-concentration of generated slates.
We also propose to separate a pivot selection phase from the generation process so that the model can apply perturbation before generation.
arXiv Detail & Related papers (2021-02-26T05:04:40Z) - Autoregressive Score Matching [113.4502004812927]
We propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariable log-conditionals (scores)
For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training.
We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.
arXiv Detail & Related papers (2020-10-24T07:01:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.