Addressing Popularity Bias in Third-Party Library Recommendations Using LLMs

• URL: http://arxiv.org/abs/2501.10313v1
• Date: Fri, 17 Jan 2025 17:35:14 GMT
• Title: Addressing Popularity Bias in Third-Party Library Recommendations Using LLMs
• Authors: Claudio Di Sipio, Juri Di Rocco, Davide Di Ruscio, Vladyslav Bulhakov,
• Abstract summary: This paper investigates the capability of large language models to address the popularity bias in recommender systems of third-party libraries (TPLs) We conduct an ablation study experimenting with state-of-the-art techniques to mitigate the popularity bias, including fine-tuning and popularity penalty mechanisms. Our findings reveal that the considered LLMs cannot address the popularity bias in TPL recommenders, even though fine-tuning and post-processing penalty mechanism contributes to increasing the overall diversity of the provided recommendations.
• Score: 6.106023882846559
• License:
• Abstract: Recommender systems for software engineering (RSSE) play a crucial role in automating development tasks by providing relevant suggestions according to the developer's context. However, they suffer from the so-called popularity bias, i.e., the phenomenon of recommending popular items that might be irrelevant to the current task. In particular, the long-tail effect can hamper the system's performance in terms of accuracy, thus leading to false positives in the provided recommendations. Foundation models are the most advanced generative AI-based models that achieve relevant results in several SE tasks. This paper aims to investigate the capability of large language models (LLMs) to address the popularity bias in recommender systems of third-party libraries (TPLs). We conduct an ablation study experimenting with state-of-the-art techniques to mitigate the popularity bias, including fine-tuning and popularity penalty mechanisms. Our findings reveal that the considered LLMs cannot address the popularity bias in TPL recommenders, even though fine-tuning and post-processing penalty mechanism contributes to increasing the overall diversity of the provided recommendations. In addition, we discuss the limitations of LLMs in this context and suggest potential improvements to address the popularity bias in TPL recommenders, thus paving the way for additional experiments in this direction.

Related papers

• Generative Large Recommendation Models: Emerging Trends in LLMs for Recommendation [85.52251362906418]
  This tutorial explores two primary approaches for integrating large language models (LLMs) It provides a comprehensive overview of generative large recommendation models, including their recent advancements, challenges, and potential research directions. Key topics include data quality, scaling laws, user behavior mining, and efficiency in training and inference.
  arXiv  Detail & Related papers  (2025-02-19T14:48:25Z)
• Correcting for Popularity Bias in Recommender Systems via Item Loss Equalization [1.7771454131646311]
  A small set of popular items dominate the recommendation results due to their high interaction rates. This phenomenon disproportionately benefits users with mainstream tastes while neglecting those with niche interests. We propose an in-processing approach to address this issue by intervening in the training process of recommendation models.
  arXiv  Detail & Related papers  (2024-10-07T08:34:18Z)
• Cognitive Biases in Large Language Models for News Recommendation [68.90354828533535]
  This paper explores the potential impact of cognitive biases on large language models (LLMs) based news recommender systems. We discuss strategies to mitigate these biases through data augmentation, prompt engineering and learning algorithms aspects.
  arXiv  Detail & Related papers  (2024-10-03T18:42:07Z)
• Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge [84.34545223897578]
  Despite their excellence in many domains, potential issues are under-explored, undermining their reliability and the scope of their utility. We identify 12 key potential biases and propose a new automated bias quantification framework-CALM- which quantifies and analyzes each type of bias in LLM-as-a-Judge. Our work highlights the need for stakeholders to address these issues and remind users to exercise caution in LLM-as-a-Judge applications.
  arXiv  Detail & Related papers  (2024-10-03T17:53:30Z)
• Incorporate LLMs with Influential Recommender System [34.5820082133773]
  proactive recommender systems recommend a sequence of items to guide user interest in the target item. Existing methods struggle to construct a coherent influence path that builds up with items the user is likely to enjoy. We introduce a novel approach named LLM-based Influence Path Planning (LLM-IPP) Our approach maintains coherence between consecutive recommendations and enhances user acceptability of the recommended items.
  arXiv  Detail & Related papers  (2024-09-07T13:41:37Z)
• Large Language Models as Recommender Systems: A Study of Popularity Bias [46.17953988777199]
  Popular items are disproportionately recommended, overshadowing less popular but potentially relevant items. Recent advancements have seen the integration of general-purpose Large Language Models into recommender systems. Our study explores whether LLMs contribute to or can alleviate popularity bias in recommender systems.
  arXiv  Detail & Related papers  (2024-06-03T12:53:37Z)
• GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models [83.30078426829627]
  Large language models (LLMs) have gained popularity and are being widely adopted by a large user community. The existing evaluation methods have many constraints, and their results exhibit a limited degree of interpretability. We propose a bias evaluation framework named GPTBIAS that leverages the high performance of LLMs to assess bias in models.
  arXiv  Detail & Related papers  (2023-12-11T12:02:14Z)
• Metrics for popularity bias in dynamic recommender systems [0.0]
  Biased recommendations may lead to decisions that can potentially have adverse effects on individuals, sensitive user groups, and society. This paper focuses on quantifying popularity bias that stems directly from the output of RecSys models. Four metrics to quantify popularity bias in RescSys over time in dynamic setting across different sensitive user groups have been proposed.
  arXiv  Detail & Related papers  (2023-10-12T16:15:30Z)
• Off-policy evaluation for learning-to-rank via interpolating the item-position model and the position-based model [83.83064559894989]
  A critical need for industrial recommender systems is the ability to evaluate recommendation policies offline, before deploying them to production. We develop a new estimator that mitigates the problems of the two most popular off-policy estimators for rankings. In particular, the new estimator, called INTERPOL, addresses the bias of a potentially misspecified position-based model.
  arXiv  Detail & Related papers  (2022-10-15T17:22:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.