Learning Transferrable Parameters for Long-tailed Sequential User Behavior Modeling

• URL: http://arxiv.org/abs/2010.11401v2
• Date: Sun, 1 Nov 2020 13:36:51 GMT
• Title: Learning Transferrable Parameters for Long-tailed Sequential User Behavior Modeling
• Authors: Jianwen Yin, Chenghao Liu, Weiqing Wang, Jianling Sun, Steven C.H. Hoi
• Abstract summary: We argue that focusing on tail users could bring more benefits and address the long tails issue. Specifically, we propose a gradient alignment and adopt an adversarial training scheme to facilitate knowledge transfer from the head to the tail.
• Score: 70.64257515361972
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sequential user behavior modeling plays a crucial role in online user-oriented services, such as product purchasing, news feed consumption, and online advertising. The performance of sequential modeling heavily depends on the scale and quality of historical behaviors. However, the number of user behaviors inherently follows a long-tailed distribution, which has been seldom explored. In this work, we argue that focusing on tail users could bring more benefits and address the long tails issue by learning transferrable parameters from both optimization and feature perspectives. Specifically, we propose a gradient alignment optimizer and adopt an adversarial training scheme to facilitate knowledge transfer from the head to the tail. Such methods can also deal with the cold-start problem of new users. Moreover, it could be directly adaptive to various well-established sequential models. Extensive experiments on four real-world datasets verify the superiority of our framework compared with the state-of-the-art baselines.

Related papers

• Prioritized Generative Replay [121.83947140497655]
  We propose a prioritized, parametric version of an agent's memory, using generative models to capture online experience. This paradigm enables densification of past experience, with new generations that benefit from the generative model's generalization capacity. We show this recipe can be instantiated using conditional diffusion models and simple relevance functions.
  arXiv  Detail & Related papers  (2024-10-23T17:59:52Z)
• Incorporating Group Prior into Variational Inference for Tail-User Behavior Modeling in CTR Prediction [8.213386595519928]
  We propose a novel variational inference approach, namely Group Prior Sampler Variational Inference (GPSVI) GPSVI introduces group preferences as priors to refine latent user interests for tail users. Rigorous analysis and extensive experiments demonstrate that GPSVI consistently improves the performance of tail users.
  arXiv  Detail & Related papers  (2024-10-19T13:15:36Z)
• AdaptSSR: Pre-training User Model with Augmentation-Adaptive Self-Supervised Ranking [19.1857792382924]
  We propose Augmentation-Supervised Ranking (AdaptSSR) to replace the contrastive learning task. We adopt a multiple pairwise ranking loss which trains the user model to capture the similarity orders between the implicitly augmented view, the explicitly augmented view, and views from other users. Experiments on both public and industrial datasets with six downstream tasks verify the effectiveness of AdaptSSR.
  arXiv  Detail & Related papers  (2023-10-15T02:19:28Z)
• Latent User Intent Modeling for Sequential Recommenders [92.66888409973495]
  Sequential recommender models learn to predict the next items a user is likely to interact with based on his/her interaction history on the platform. Most sequential recommenders however lack a higher-level understanding of user intents, which often drive user behaviors online. Intent modeling is thus critical for understanding users and optimizing long-term user experience.
  arXiv  Detail & Related papers  (2022-11-17T19:00:24Z)
• Personalizing Intervened Network for Long-tailed Sequential User Behavior Modeling [66.02953670238647]
  Tail users suffer from significantly lower-quality recommendation than the head users after joint training. A model trained on tail users separately still achieve inferior results due to limited data. We propose a novel approach that significantly improves the recommendation performance of the tail users.
  arXiv  Detail & Related papers  (2022-08-19T02:50:19Z)
• Sequential Search with Off-Policy Reinforcement Learning [48.88165680363482]
  We propose a highly scalable hybrid learning model that consists of an RNN learning framework and an attention model. As a novel optimization step, we fit multiple short user sequences in a single RNN pass within a training batch, by solving a greedy knapsack problem on the fly. We also explore the use of off-policy reinforcement learning in multi-session personalized search ranking.
  arXiv  Detail & Related papers  (2022-02-01T06:52:40Z)
• How Well Do Sparse Imagenet Models Transfer? [75.98123173154605]
  Transfer learning is a classic paradigm by which models pretrained on large "upstream" datasets are adapted to yield good results on "downstream" datasets. In this work, we perform an in-depth investigation of this phenomenon in the context of convolutional neural networks (CNNs) trained on the ImageNet dataset. We show that sparse models can match or even outperform the transfer performance of dense models, even at high sparsities.
  arXiv  Detail & Related papers  (2021-11-26T11:58:51Z)
• Handling Long-Tail Queries with Slice-Aware Conversational Systems [29.41747850530487]
  We explore the recent concept of slice-based learning (SBL) We first define a set of labeling functions to generate weak supervision data for the tail intents. We then extend the baseline model towards a slice, which monitors and improves the model performance on the selected tail intents. Experiments show that the slice-aware model is beneficial in improving model performance for the tail intents while maintaining the overall performance.
  arXiv  Detail & Related papers  (2021-04-26T10:23:28Z)
• Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation [111.44445634272235]
  In this paper, we develop a parameter efficient transfer learning architecture, termed as PeterRec. PeterRec allows the pre-trained parameters to remain unaltered during fine-tuning by injecting a series of re-learned neural networks. We perform extensive experimental ablation to show the effectiveness of the learned user representation in five downstream tasks.
  arXiv  Detail & Related papers  (2020-01-13T14:09:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.