The Fault in Our Recommendations: On the Perils of Optimizing the Measurable

• URL: http://arxiv.org/abs/2405.03948v1
• Date: Tue, 07 May 2024 02:12:17 GMT
• Title: The Fault in Our Recommendations: On the Perils of Optimizing the Measurable
• Authors: Omar Besbes, Yash Kanoria, Akshit Kumar,
• Abstract summary: We show that optimizing for engagement can lead to significant utility losses. We propose a utility-aware policy that initially recommends a mix of popular and niche content.
• Score: 2.6217304977339473
• License:
• Abstract: Recommendation systems are widespread, and through customized recommendations, promise to match users with options they will like. To that end, data on engagement is collected and used. Most recommendation systems are ranking-based, where they rank and recommend items based on their predicted engagement. However, the engagement signals are often only a crude proxy for utility, as data on the latter is rarely collected or available. This paper explores the following question: By optimizing for measurable proxies, are recommendation systems at risk of significantly under-delivering on utility? If so, how can one improve utility which is seldom measured? To study these questions, we introduce a model of repeated user consumption in which, at each interaction, users select between an outside option and the best option from a recommendation set. Our model accounts for user heterogeneity, with the majority preferring ``popular'' content, and a minority favoring ``niche'' content. The system initially lacks knowledge of individual user preferences but can learn them through observations of users' choices over time. Our theoretical and numerical analysis demonstrate that optimizing for engagement can lead to significant utility losses. Instead, we propose a utility-aware policy that initially recommends a mix of popular and niche content. As the platform becomes more forward-looking, our utility-aware policy achieves the best of both worlds: near-optimal utility and near-optimal engagement simultaneously. Our study elucidates an important feature of recommendation systems; given the ability to suggest multiple items, one can perform significant exploration without incurring significant reductions in engagement. By recommending high-risk, high-reward items alongside popular items, systems can enhance discovery of high utility items without significantly affecting engagement.

Related papers

• An Efficient Multi-threaded Collaborative Filtering Approach in Recommendation System [0.0]
  This research focuses on building a scalable recommendation system capable of handling numerous users efficiently. A multithreaded similarity approach is employed to achieve this, where users are divided into independent threads that run in parallel. This parallelization significantly reduces computation time compared to traditional methods, resulting in a faster, more efficient, and scalable recommendation system.
  arXiv  Detail & Related papers  (2024-09-28T06:33:18Z)
• Relevance meets Diversity: A User-Centric Framework for Knowledge Exploration through Recommendations [15.143224593682012]
  We propose a novel recommendation strategy that combines relevance and diversity by a copula function. We use diversity as a surrogate of the amount of knowledge obtained by the user while interacting with the system. Our strategy outperforms several state-of-the-art competitors.
  arXiv  Detail & Related papers  (2024-08-07T13:48:24Z)
• The MovieLens Beliefs Dataset: Collecting Pre-Choice Data for Online Recommender Systems [0.0]
  This paper introduces a method for collecting user beliefs about unexperienced items - a critical predictor of choice behavior. We implement this method on the MovieLens platform, resulting in a rich dataset that combines user ratings, beliefs, and observed recommendations.
  arXiv  Detail & Related papers  (2024-05-17T19:06:06Z)
• A First Look at Selection Bias in Preference Elicitation for Recommendation [64.44255178199846]
  We study the effect of selection bias in preference elicitation on the resulting recommendations. A big hurdle is the lack of any publicly available dataset that has preference elicitation interactions. We propose a simulation of a topic-based preference elicitation process.
  arXiv  Detail & Related papers  (2024-05-01T14:56:56Z)
• Improving Recommendation System Serendipity Through Lexicase Selection [53.57498970940369]
  We propose a new serendipity metric to measure the presence of echo chambers and homophily in recommendation systems. We then attempt to improve the diversity-preservation qualities of well known recommendation techniques by adopting a parent selection algorithm known as lexicase selection. Our results show that lexicase selection, or a mixture of lexicase selection and ranking, outperforms its purely ranked counterparts in terms of personalization, coverage and our specifically designed serendipity benchmark.
  arXiv  Detail & Related papers  (2023-05-18T15:37:38Z)
• Recommendation Systems with Distribution-Free Reliability Guarantees [83.80644194980042]
  We show how to return a set of items rigorously guaranteed to contain mostly good items. Our procedure endows any ranking model with rigorous finite-sample control of the false discovery rate. We evaluate our methods on the Yahoo! Learning to Rank and MSMarco datasets.
  arXiv  Detail & Related papers  (2022-07-04T17:49:25Z)
• A Review on Pushing the Limits of Baseline Recommendation Systems with the integration of Opinion Mining & Information Retrieval Techniques [0.0]
  Recommendation Systems allow users to identify trending items among a community while being timely and relevant to the user's expectations. Deep Learning methods have been brought forward to achieve better quality recommendations. Researchers have tried to expand on the capabilities of standard recommendation systems to provide the most effective recommendations.
  arXiv  Detail & Related papers  (2022-05-03T22:13:33Z)
• Introducing a Framework and a Decision Protocol to Calibrate Recommender Systems [0.0]
  This paper proposes an approach to create recommendation lists with a calibrated balance of genres. The main claim is that calibration can contribute positively to generate fairer recommendations. We propose a conceptual framework and a decision protocol to generate more than one thousand combinations of calibrated systems.
  arXiv  Detail & Related papers  (2022-04-07T19:30:55Z)
• PURS: Personalized Unexpected Recommender System for Improving User Satisfaction [76.98616102965023]
  We describe a novel Personalized Unexpected Recommender System (PURS) model that incorporates unexpectedness into the recommendation process. Extensive offline experiments on three real-world datasets illustrate that the proposed PURS model significantly outperforms the state-of-the-art baseline approaches.
  arXiv  Detail & Related papers  (2021-06-05T01:33:21Z)
• Fairness-Aware Explainable Recommendation over Knowledge Graphs [73.81994676695346]
  We analyze different groups of users according to their level of activity, and find that bias exists in recommendation performance between different groups. We show that inactive users may be more susceptible to receiving unsatisfactory recommendations, due to insufficient training data for the inactive users. We propose a fairness constrained approach via re-ranking to mitigate this problem in the context of explainable recommendation over knowledge graphs.
  arXiv  Detail & Related papers  (2020-06-03T05:04:38Z)
• Survey for Trust-aware Recommender Systems: A Deep Learning Perspective [48.2733163413522]
  It becomes critical to embrace a trustworthy recommender system. This survey provides a systemic summary of three categories of trust-aware recommender systems.
  arXiv  Detail & Related papers  (2020-04-08T02:11:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.