Understanding or Manipulation: Rethinking Online Performance Gains of Modern Recommender Systems

• URL: http://arxiv.org/abs/2210.05662v2
• Date: Mon, 18 Dec 2023 14:13:03 GMT
• Title: Understanding or Manipulation: Rethinking Online Performance Gains of Modern Recommender Systems
• Authors: Zhengbang Zhu, Rongjun Qin, Junjie Huang, Xinyi Dai, Yang Yu, Yong Yu and Weinan Zhang
• Abstract summary: We present a framework for benchmarking the degree of manipulations of recommendation algorithms. We find that a high online click-through rate does not necessarily mean a better understanding of user initial preference. We advocate that future recommendation algorithm studies should be treated as an optimization problem with constrained user preference manipulations.
• Score: 38.75457258877731
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Recommender systems are expected to be assistants that help human users find relevant information automatically without explicit queries. As recommender systems evolve, increasingly sophisticated learning techniques are applied and have achieved better performance in terms of user engagement metrics such as clicks and browsing time. The increase in the measured performance, however, can have two possible attributions: a better understanding of user preferences, and a more proactive ability to utilize human bounded rationality to seduce user over-consumption. A natural following question is whether current recommendation algorithms are manipulating user preferences. If so, can we measure the manipulation level? In this paper, we present a general framework for benchmarking the degree of manipulations of recommendation algorithms, in both slate recommendation and sequential recommendation scenarios. The framework consists of four stages, initial preference calculation, training data collection, algorithm training and interaction, and metrics calculation that involves two proposed metrics. We benchmark some representative recommendation algorithms in both synthetic and real-world datasets under the proposed framework. We have observed that a high online click-through rate does not necessarily mean a better understanding of user initial preference, but ends in prompting users to choose more documents they initially did not favor. Moreover, we find that the training data have notable impacts on the manipulation degrees, and algorithms with more powerful modeling abilities are more sensitive to such impacts. The experiments also verified the usefulness of the proposed metrics for measuring the degree of manipulations. We advocate that future recommendation algorithm studies should be treated as an optimization problem with constrained user preference manipulations.

Related papers

• Measuring Strategization in Recommendation: Users Adapt Their Behavior to Shape Future Content [66.71102704873185]
  We test for user strategization by conducting a lab experiment and survey. We find strong evidence of strategization across outcome metrics, including participants' dwell time and use of "likes" Our findings suggest that platforms cannot ignore the effect of their algorithms on user behavior.
  arXiv  Detail & Related papers  (2024-05-09T07:36:08Z)
• Meta-Wrapper: Differentiable Wrapping Operator for User Interest Selection in CTR Prediction [97.99938802797377]
  Click-through rate (CTR) prediction, whose goal is to predict the probability of the user to click on an item, has become increasingly significant in recommender systems. Recent deep learning models with the ability to automatically extract the user interest from his/her behaviors have achieved great success. We propose a novel approach under the framework of the wrapper method, which is named Meta-Wrapper.
  arXiv  Detail & Related papers  (2022-06-28T03:28:15Z)
• On the Generalizability and Predictability of Recommender Systems [33.46314108814183]
  We give the first large-scale study of recommender system approaches. We create Reczilla, a meta-learning approach to recommender systems.
  arXiv  Detail & Related papers  (2022-06-23T17:51:42Z)
• Batch versus Sequential Active Learning for Recommender Systems [3.7796614675664397]
  We show that sequential mode produces the most accurate recommendations for dense data sets. For most active learners, the best predictor turned out to be FunkSVD in combination with sequential mode.
  arXiv  Detail & Related papers  (2022-01-19T12:50:36Z)
• Recommending with Recommendations [1.1602089225841632]
  Recommendation systems often draw upon sensitive user information in making predictions. We show how to address this deficiency by basing a service's recommendation engine upon recommendations from other existing services. In our setting, the user's (potentially sensitive) information belongs to a high-dimensional latent space.
  arXiv  Detail & Related papers  (2021-12-02T04:30:15Z)
• D2RLIR : an improved and diversified ranking function in interactive recommendation systems based on deep reinforcement learning [0.3058685580689604]
  This paper proposes a deep reinforcement learning based recommendation system by utilizing Actor-Critic architecture. The proposed model is able to generate a diverse while relevance recommendation list based on the user's preferences.
  arXiv  Detail & Related papers  (2021-10-28T13:11:29Z)
• Learning to Learn a Cold-start Sequential Recommender [70.5692886883067]
  The cold-start recommendation is an urgent problem in contemporary online applications. We propose a meta-learning based cold-start sequential recommendation framework called metaCSR. metaCSR holds the ability to learn the common patterns from regular users' behaviors.
  arXiv  Detail & Related papers  (2021-10-18T08:11:24Z)
• Do Offline Metrics Predict Online Performance in Recommender Systems? [79.48653445643865]
  We investigate the extent to which offline metrics predict online performance by evaluating recommenders across six simulated environments. We observe that offline metrics are correlated with online performance over a range of environments. We study the impact of adding exploration strategies, and observe that their effectiveness, when compared to greedy recommendation, is highly dependent on the recommendation algorithm.
  arXiv  Detail & Related papers  (2020-11-07T01:41:13Z)
• AdaS: Adaptive Scheduling of Stochastic Gradients [50.80697760166045]
  We introduce the notions of textit"knowledge gain" and textit"mapping condition" and propose a new algorithm called Adaptive Scheduling (AdaS) Experimentation reveals that, using the derived metrics, AdaS exhibits: (a) faster convergence and superior generalization over existing adaptive learning methods; and (b) lack of dependence on a validation set to determine when to stop training.
  arXiv  Detail & Related papers  (2020-06-11T16:36:31Z)
• Recommendation system using a deep learning and graph analysis approach [1.2183405753834562]
  We propose a novel recommendation method based on Matrix Factorization and graph analysis methods. In addition, we leverage deep Autoencoders to initialize users and items latent factors, and deep embedding method gathers users' latent factors from the user trust graph.
  arXiv  Detail & Related papers  (2020-04-17T08:05:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.