User Profile with Large Language Models: Construction, Updating, and Benchmarking

• URL: http://arxiv.org/abs/2502.10660v1
• Date: Sat, 15 Feb 2025 03:57:52 GMT
• Title: User Profile with Large Language Models: Construction, Updating, and Benchmarking
• Authors: Nusrat Jahan Prottasha, Md Kowsher, Hafijur Raman, Israt Jahan Anny, Prakash Bhat, Ivan Garibay, Ozlem Garibay,
• Abstract summary: We present two high-quality open-source user profile datasets. These datasets offer a strong basis for evaluating user profile modeling techniques. We show a methodology that uses large language models to tackle both profile construction and updating.
• Score: 0.3350491650545292
• License:
• Abstract: User profile modeling plays a key role in personalized systems, as it requires building accurate profiles and updating them with new information. In this paper, we present two high-quality open-source user profile datasets: one for profile construction and another for profile updating. These datasets offer a strong basis for evaluating user profile modeling techniques in dynamic settings. We also show a methodology that uses large language models (LLMs) to tackle both profile construction and updating. Our method uses a probabilistic framework to predict user profiles from input text, allowing for precise and context-aware profile generation. Our experiments demonstrate that models like Mistral-7b and Llama2-7b perform strongly in both tasks. LLMs improve the precision and recall of the generated profiles, and high evaluation scores confirm the effectiveness of our approach.

Related papers

• Automated Personnel Selection for Software Engineers Using LLM-Based Profile Evaluation [1.0718353079920009]
  This work evaluates software engineer profiles using an automated staff selection method based on advanced natural language processing (NLP) techniques. A fresh dataset was generated by collecting LinkedIn profiles with important attributes like education, experience, skills, and self-introduction. With 85% accuracy and an F1 score of 0.85, RoBERTa performed the best; DistilBERT provided comparable results at less computing expense.
  arXiv  Detail & Related papers  (2024-10-30T18:12:19Z)
• Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration [90.41908331897639]
  Large language models (LLMs) have significantly benefited from training on diverse, high-quality task-specific data. We present a novel approach, ReverseGen, designed to automatically generate effective training samples.
  arXiv  Detail & Related papers  (2024-10-22T06:43:28Z)
• Prompt Tuning as User Inherent Profile Inference Machine [53.78398656789463]
  We propose UserIP-Tuning, which uses prompt-tuning to infer user profiles. A profile quantization codebook bridges the modality gap by profile embeddings into collaborative IDs. Experiments on four public datasets show that UserIP-Tuning outperforms state-of-the-art recommendation algorithms.
  arXiv  Detail & Related papers  (2024-08-13T02:25:46Z)
• Understanding the Role of User Profile in the Personalization of Large Language Models [19.74964898049076]
  This study first confirms that the effectiveness of user profiles is primarily due to personalization information rather than semantic information. Within the user profile, it is the historical personalized response produced or approved by users that plays a pivotal role in personalizing LLMs. Our findings reveal the role of user profiles for the personalization of LLMs, and showcase how incorporating user profiles impacts performance.
  arXiv  Detail & Related papers  (2024-06-22T14:32:35Z)
• Step-Back Profiling: Distilling User History for Personalized Scientific Writing [50.481041470669766]
  Large language models (LLM) excel at a variety of natural language processing tasks, yet they struggle to generate personalized content for individuals. We introduce STEP-BACK PROFILING to personalize LLMs by distilling user history into concise profiles. Our approach outperforms the baselines by up to 3.6 points on the general personalization benchmark.
  arXiv  Detail & Related papers  (2024-06-20T12:58:26Z)
• PRODIGy: a PROfile-based DIalogue Generation dataset [12.566555088877871]
  We propose a new resource where each dialogue is aligned with all possible speaker representations such as communication style, biographies, and personality. This framework allows to test several baselines built using generative language models with several profile configurations.
  arXiv  Detail & Related papers  (2023-11-09T08:19:34Z)
• Visual Prompt Based Personalized Federated Learning [83.04104655903846]
  We propose a novel PFL framework for image classification tasks, dubbed pFedPT, that leverages personalized visual prompts to implicitly represent local data distribution information of clients. Experiments on the CIFAR10 and CIFAR100 datasets show that pFedPT outperforms several state-of-the-art (SOTA) PFL algorithms by a large margin in various settings.
  arXiv  Detail & Related papers  (2023-03-15T15:02:15Z)
• A Cooperative Memory Network for Personalized Task-oriented Dialogue Systems with Incomplete User Profiles [55.951126447217526]
  We study personalized Task-oriented Dialogue Systems without assuming that user profiles are complete. We propose a Cooperative Memory Network (CoMemNN) that has a novel mechanism to gradually enrich user profiles. CoMemNN is able to enrich user profiles effectively, which results in an improvement of 3.06% in terms of response selection accuracy.
  arXiv  Detail & Related papers  (2021-02-16T18:05:54Z)
• When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data [84.87772675171412]
  We study the circumstances under which explanations of individual data points can improve modeling performance. We make use of three existing datasets with explanations: e-SNLI, TACRED, SemEval.
  arXiv  Detail & Related papers  (2021-02-03T18:57:08Z)
• Reinforced Imitative Graph Representation Learning for Mobile User Profiling: An Adversarial Training Perspective [21.829562421373712]
  We study the problem of mobile user profiling, which is a critical component for quantifying users' characteristics in the human mobility modeling pipeline. We propose an imitation-based mobile user profiling framework by exploiting reinforcement learning.
  arXiv  Detail & Related papers  (2021-01-07T17:10:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.