Incremental user embedding modeling for personalized text classification

• URL: http://arxiv.org/abs/2202.06369v1
• Date: Sun, 13 Feb 2022 17:33:35 GMT
• Title: Incremental user embedding modeling for personalized text classification
• Authors: Ruixue Lian, Che-Wei Huang, Yuqing Tang, Qilong Gu, Chengyuan Ma, Chenlei Guo
• Abstract summary: Individual user profiles and interaction histories play a significant role in providing customized experiences in real-world applications. We propose an incremental user embedding modeling approach, in which embeddings of user's recent interaction histories are dynamically integrated into the accumulated history vectors. We demonstrate the effectiveness of this approach by applying it to a personalized multi-class classification task based on the Reddit dataset.
• Score: 12.381095398791352
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Individual user profiles and interaction histories play a significant role in providing customized experiences in real-world applications such as chatbots, social media, retail, and education. Adaptive user representation learning by utilizing user personalized information has become increasingly challenging due to ever-growing history data. In this work, we propose an incremental user embedding modeling approach, in which embeddings of user's recent interaction histories are dynamically integrated into the accumulated history vectors via a transformer encoder. This modeling paradigm allows us to create generalized user representations in a consecutive manner and also alleviate the challenges of data management. We demonstrate the effectiveness of this approach by applying it to a personalized multi-class classification task based on the Reddit dataset, and achieve 9% and 30% relative improvement on prediction accuracy over a baseline system for two experiment settings through appropriate comment history encoding and task modeling.

Related papers

• USE: Dynamic User Modeling with Stateful Sequence Models [26.74966828348815]
  User Stateful Embedding (USE) generates user embeddings without the need for exhaustive reprocessing. We introduce a novel training objective named future W-behavior prediction to transcend the limitations of next-token prediction. We conduct experiments on 8 downstream tasks using Snapchat users' behavioral logs in both static (i.e., fixed user behavior sequences) and dynamic (i.e. periodically updated user behavior sequences) settings.
  arXiv  Detail & Related papers  (2024-03-20T07:05:19Z)
• Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyond [87.1712108247199]
  Our goal is to establish a Unified paradigm for Multi-modal Personalization systems (UniMP) We develop a generic and personalization generative framework, that can handle a wide range of personalized needs. Our methodology enhances the capabilities of foundational language models for personalized tasks.
  arXiv  Detail & Related papers  (2024-03-15T20:21:31Z)
• Generalized User Representations for Transfer Learning [6.953653891411339]
  We present a novel framework for user representation in large-scale recommender systems. Our approach employs a two-stage methodology combining representation learning and transfer learning. We show how the proposed framework can significantly reduce infrastructure costs compared to alternative approaches.
  arXiv  Detail & Related papers  (2024-03-01T15:05:21Z)
• Persona-DB: Efficient Large Language Model Personalization for Response Prediction with Collaborative Data Refinement [79.2400720115588]
  We introduce Persona-DB, a simple yet effective framework consisting of a hierarchical construction process to improve generalization across task contexts. In the evaluation of response prediction, Persona-DB demonstrates superior context efficiency in maintaining accuracy with a significantly reduced retrieval size. Our experiments also indicate a marked improvement of over 10% under cold-start scenarios, when users have extremely sparse data.
  arXiv  Detail & Related papers  (2024-02-16T20:20:43Z)
• Unsupervised Neural Stylistic Text Generation using Transfer learning and Adapters [66.17039929803933]
  We propose a novel transfer learning framework which updates only $0.3%$ of model parameters to learn style specific attributes for response generation. We learn style specific attributes from the PERSONALITY-CAPTIONS dataset.
  arXiv  Detail & Related papers  (2022-10-07T00:09:22Z)
• PinnerFormer: Sequence Modeling for User Representation at Pinterest [60.335384724891746]
  We introduce PinnerFormer, a user representation trained to predict a user's future long-term engagement. Unlike prior approaches, we adapt our modeling to a batch infrastructure via our new dense all-action loss. We show that by doing so, we significantly close the gap between batch user embeddings that are generated once a day and realtime user embeddings generated whenever a user takes an action.
  arXiv  Detail & Related papers  (2022-05-09T18:26:51Z)
• UserBERT: Modeling Long- and Short-Term User Preferences via Self-Supervision [6.8904125699168075]
  This paper extends the BERT model to e-commerce user data for pre-training representations in a self-supervised manner. By viewing user actions in sequences as analogous to words in sentences, we extend the existing BERT model to user behavior data. We propose methods for the tokenization of different types of user behavior sequences, the generation of input representation, and a novel pretext task to enable the pre-trained model to learn from its own input.
  arXiv  Detail & Related papers  (2022-02-14T08:31:36Z)
• Exploiting Behavioral Consistence for Universal User Representation [11.290137806288191]
  We focus on developing universal user representation model. The obtained universal representations are expected to contain rich information. We propose Self-supervised User Modeling Network (SUMN) to encode behavior data into the universal representation.
  arXiv  Detail & Related papers  (2020-12-11T06:10:14Z)
• Learning Transferrable Parameters for Long-tailed Sequential User Behavior Modeling [70.64257515361972]
  We argue that focusing on tail users could bring more benefits and address the long tails issue. Specifically, we propose a gradient alignment and adopt an adversarial training scheme to facilitate knowledge transfer from the head to the tail.
  arXiv  Detail & Related papers  (2020-10-22T03:12:02Z)
• Unsupervised Model Personalization while Preserving Privacy and Scalability: An Open Problem [55.21502268698577]
  This work investigates the task of unsupervised model personalization, adapted to continually evolving, unlabeled local user images. We provide a novel Dual User-Adaptation framework (DUA) to explore the problem. This framework flexibly disentangles user-adaptation into model personalization on the server and local data regularization on the user device.
  arXiv  Detail & Related papers  (2020-03-30T09:35:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.