Biases in LLM-Generated Musical Taste Profiles for Recommendation

• URL: http://arxiv.org/abs/2507.16708v1
• Date: Tue, 22 Jul 2025 15:44:10 GMT
• Title: Biases in LLM-Generated Musical Taste Profiles for Recommendation
• Authors: Bruno Sguerra, Elena V. Epure, Harin Lee, Manuel Moussallam,
• Abstract summary: Large Language Models (LLMs) for recommendation can generate Natural Language (NL) user taste profiles from consumption data.<n>But it remains unclear whether users consider these profiles to be an accurate representation of their taste.<n>We study this issue in the context of music streaming, where personalization is challenged by a large and culturally diverse catalog.
• Score: 6.482557558168364
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One particularly promising use case of Large Language Models (LLMs) for recommendation is the automatic generation of Natural Language (NL) user taste profiles from consumption data. These profiles offer interpretable and editable alternatives to opaque collaborative filtering representations, enabling greater transparency and user control. However, it remains unclear whether users consider these profiles to be an accurate representation of their taste, which is crucial for trust and usability. Moreover, because LLMs inherit societal and data-driven biases, profile quality may systematically vary across user and item characteristics. In this paper, we study this issue in the context of music streaming, where personalization is challenged by a large and culturally diverse catalog. We conduct a user study in which participants rate NL profiles generated from their own listening histories. We analyze whether identification with the profiles is biased by user attributes (e.g., mainstreamness, taste diversity) and item features (e.g., genre, country of origin). We also compare these patterns to those observed when using the profiles in a downstream recommendation task. Our findings highlight both the potential and limitations of scrutable, LLM-based profiling in personalized systems.

Related papers

• LettinGo: Explore User Profile Generation for Recommendation System [45.40232561275015]
  We introduce LettinGo, a novel framework for generating diverse and adaptive user profiles.<n>Our framework significantly enhances recommendation accuracy, flexibility, and contextual awareness.
  arXiv  Detail & Related papers  (2025-06-23T05:51:52Z)
• NextQuill: Causal Preference Modeling for Enhancing LLM Personalization [82.15961484963256]
  We introduce NextQuill, a novel personalization framework grounded in causal preference modeling.<n>Building on this insight, NextQuill introduces two complementary alignment strategies.<n> Experiments across multiple personalization benchmarks demonstrate that NextQuill significantly improves personalization quality.
  arXiv  Detail & Related papers  (2025-06-03T02:08:55Z)
• Towards Explainable Temporal User Profiling with LLMs [3.719862246745416]
  We leverage large language models (LLMs) to generate natural language summaries of users' interaction histories.<n>Our framework not only models temporal user preferences but also produces natural language profiles that can be used to explain recommendations in an interpretable manner.
  arXiv  Detail & Related papers  (2025-05-01T22:02:46Z)
• Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale [51.9706400130481]
  Large Language Models (LLMs) have emerged as personalized assistants for users across a wide range of tasks.<n> PERSONAMEM features curated user profiles with over 180 simulated user-LLM interaction histories.<n>We evaluate LLM chatbots' ability to identify the most suitable response according to the current state of the user's profile.
  arXiv  Detail & Related papers  (2025-04-19T08:16:10Z)
• Optimizing Data Delivery: Insights from User Preferences on Visuals, Tables, and Text [59.68239795065175]
  We conduct a user study where users are shown a question and asked what they would prefer to see. We use the data to establish that a user's personal traits does influence the data outputs that they prefer.
  arXiv  Detail & Related papers  (2024-11-12T00:24:31Z)
• Stereotype or Personalization? User Identity Biases Chatbot Recommendations [65.10885980789293]
  We show that models generate racially stereotypical recommendations regardless of whether the user revealed their identity intentionally.<n>We argue that chatbots ought to transparently indicate when recommendations are influenced by a user's revealed identity characteristics, but observe that they currently fail to do so.
  arXiv  Detail & Related papers  (2024-10-08T01:51:55Z)
• PersonalLLM: Tailoring LLMs to Individual Preferences [11.717169516971856]
  We present a public benchmark, PersonalLLM, focusing on adapting LLMs to provide maximal benefits for a particular user.<n>We curate open-ended prompts paired with many high-quality answers over which users would be expected to display heterogeneous latent preferences.<n>Our dataset and generated personalities offer an innovative testbed for developing personalization algorithms.
  arXiv  Detail & Related papers  (2024-09-30T13:55:42Z)
• Guided Profile Generation Improves Personalization with LLMs [3.2685922749445617]
  In modern commercial systems, including Recommendation, Ranking, and E-Commerce platforms, there is a trend towards incorporating Personalization context as input into Large Language Models (LLMs) We propose Guided Profile Generation (GPG), a general method designed to generate personal profiles in natural language. Our experimental results show that GPG improves LLM's personalization ability across different tasks, for example, it increases 37% accuracy in predicting personal preference compared to directly feeding the LLMs with raw personal context.
  arXiv  Detail & Related papers  (2024-09-19T21:29:56Z)
• Understanding the Role of User Profile in the Personalization of Large Language Models [19.74964898049076]
  This study first confirms that the effectiveness of user profiles is primarily due to personalization information rather than semantic information. Within the user profile, it is the historical personalized response produced or approved by users that plays a pivotal role in personalizing LLMs. Our findings reveal the role of user profiles for the personalization of LLMs, and showcase how incorporating user profiles impacts performance.
  arXiv  Detail & Related papers  (2024-06-22T14:32:35Z)
• Step-Back Profiling: Distilling User History for Personalized Scientific Writing [50.481041470669766]
  Large language models (LLM) excel at a variety of natural language processing tasks, yet they struggle to generate personalized content for individuals. We introduce STEP-BACK PROFILING to personalize LLMs by distilling user history into concise profiles. Our approach outperforms the baselines by up to 3.6 points on the general personalization benchmark.
  arXiv  Detail & Related papers  (2024-06-20T12:58:26Z)
• A Cooperative Memory Network for Personalized Task-oriented Dialogue Systems with Incomplete User Profiles [55.951126447217526]
  We study personalized Task-oriented Dialogue Systems without assuming that user profiles are complete. We propose a Cooperative Memory Network (CoMemNN) that has a novel mechanism to gradually enrich user profiles. CoMemNN is able to enrich user profiles effectively, which results in an improvement of 3.06% in terms of response selection accuracy.
  arXiv  Detail & Related papers  (2021-02-16T18:05:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.