Beyond Static Calibration: The Impact of User Preference Dynamics on Calibrated Recommendation

• URL: http://arxiv.org/abs/2405.10232v1
• Date: Thu, 16 May 2024 16:33:34 GMT
• Title: Beyond Static Calibration: The Impact of User Preference Dynamics on Calibrated Recommendation
• Authors: Kun Lin, Masoud Mansoury, Farzad Eskandanian, Milad Sabouri, Bamshad Mobasher,
• Abstract summary: Calibration in recommender systems is an important performance criterion. Standard methods for mitigating miscalibration typically assume that user preference profiles are static. We conjecture that this approach can lead to recommendations that, while appearing calibrated, in fact, distort users' true preferences.
• Score: 3.324986723090369
• License:
• Abstract: Calibration in recommender systems is an important performance criterion that ensures consistency between the distribution of user preference categories and that of recommendations generated by the system. Standard methods for mitigating miscalibration typically assume that user preference profiles are static, and they measure calibration relative to the full history of user's interactions, including possibly outdated and stale preference categories. We conjecture that this approach can lead to recommendations that, while appearing calibrated, in fact, distort users' true preferences. In this paper, we conduct a preliminary investigation of recommendation calibration at a more granular level, taking into account evolving user preferences. By analyzing differently sized training time windows from the most recent interactions to the oldest, we identify the most relevant segment of user's preferences that optimizes the calibration metric. We perform an exploratory analysis with datasets from different domains with distinctive user-interaction characteristics. We demonstrate how the evolving nature of user preferences affects recommendation calibration, and how this effect is manifested differently depending on the characteristics of the data in a given domain. Datasets, codes, and more detailed experimental results are available at: https://github.com/nicolelin13/DynamicCalibrationUMAP.

Related papers

• Context-aware adaptive personalised recommendation: a meta-hybrid [0.41436032949434404]
  We propose a meta-hybrid recommender that uses machine learning to predict an optimal algorithm. Based on the proposed model, it is possible to predict which recommender will provide the most precise recommendations to a user.
  arXiv  Detail & Related papers  (2024-10-17T09:24:40Z)
• Quantifying User Coherence: A Unified Framework for Cross-Domain Recommendation Analysis [69.37718774071793]
  This paper introduces novel information-theoretic measures for understanding recommender systems. We evaluate 7 recommendation algorithms across 9 datasets, revealing the relationships between our measures and standard performance metrics.
  arXiv  Detail & Related papers  (2024-10-03T13:02:07Z)
• Algorithmic Drift: A Simulation Framework to Study the Effects of Recommender Systems on User Preferences [7.552217586057245]
  We propose a simulation framework that mimics user-recommender system interactions in a long-term scenario. We introduce two novel metrics for quantifying the algorithm's impact on user preferences, specifically in terms of drift over time.
  arXiv  Detail & Related papers  (2024-09-24T21:54:22Z)
• Rank-Preference Consistency as the Appropriate Metric for Recommender Systems [4.3166389349316425]
  We argue that unitary-invariant measures of recommender system (RS) performance fail to assess fundamental RS properties. We propose rank-preference consistency, which simply counts the number of prediction pairs that are inconsistent with the user's expressed product preferences.
  arXiv  Detail & Related papers  (2024-04-26T01:11:07Z)
• Separating and Learning Latent Confounders to Enhancing User Preferences Modeling [6.0853798070913845]
  We propose a novel framework, Separating and Learning Latent Confounders For Recommendation (SLFR) SLFR obtains the representation of unmeasured confounders to identify the counterfactual feedback by disentangling user preferences and unmeasured confounders. Experiments in five real-world datasets validate the advantages of our method.
  arXiv  Detail & Related papers  (2023-11-02T08:42:50Z)
• Meta-Wrapper: Differentiable Wrapping Operator for User Interest Selection in CTR Prediction [97.99938802797377]
  Click-through rate (CTR) prediction, whose goal is to predict the probability of the user to click on an item, has become increasingly significant in recommender systems. Recent deep learning models with the ability to automatically extract the user interest from his/her behaviors have achieved great success. We propose a novel approach under the framework of the wrapper method, which is named Meta-Wrapper.
  arXiv  Detail & Related papers  (2022-06-28T03:28:15Z)
• Estimating and Penalizing Induced Preference Shifts in Recommender Systems [10.052697877248601]
  We argue that system designers should: estimate the shifts a recommender would induce; evaluate whether such shifts would be undesirable; and even actively optimize to avoid problematic shifts. We do this by using historical user interaction data to train predictive user model which implicitly contains their preference dynamics. In simulated experiments, we show that our learned preference dynamics model is effective in estimating user preferences and how they would respond to new recommenders.
  arXiv  Detail & Related papers  (2022-04-25T21:04:46Z)
• Modeling Dynamic User Preference via Dictionary Learning for Sequential Recommendation [133.8758914874593]
  Capturing the dynamics in user preference is crucial to better predict user future behaviors because user preferences often drift over time. Many existing recommendation algorithms -- including both shallow and deep ones -- often model such dynamics independently. This paper considers the problem of embedding a user's sequential behavior into the latent space of user preferences.
  arXiv  Detail & Related papers  (2022-04-02T03:23:46Z)
• Control Variates for Slate Off-Policy Evaluation [112.35528337130118]
  We study the problem of off-policy evaluation from batched contextual bandit data with multidimensional actions. We obtain new estimators with risk improvement guarantees over both the PI and self-normalized PI estimators.
  arXiv  Detail & Related papers  (2021-06-15T06:59:53Z)
• Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits [55.59198568303196]
  This work presents LineCoSpar, a human-in-the-loop preference-based framework to learn user preferences in high dimensions. In simulations and human trials, we empirically verify that LineCoSpar is a sample-efficient approach for high-dimensional preference optimization. This result has implications for exoskeleton gait synthesis, an active field with applications to clinical use and patient rehabilitation.
  arXiv  Detail & Related papers  (2020-03-13T22:02:15Z)
• MetaSelector: Meta-Learning for Recommendation with User-Level Adaptive Model Selection [110.87712780017819]
  We propose a meta-learning framework to facilitate user-level adaptive model selection in recommender systems. We conduct experiments on two public datasets and a real-world production dataset.
  arXiv  Detail & Related papers  (2020-01-22T16:05:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.