The Bandit's Blind Spot: The Critical Role of User State Representation in Recommender Systems

• URL: http://arxiv.org/abs/2604.26651v1
• Date: Wed, 29 Apr 2026 13:18:48 GMT
• Title: The Bandit's Blind Spot: The Critical Role of User State Representation in Recommender Systems
• Abstract summary: We investigate the impact of different embedding-based state representations on the performance of traditional CMAB algorithms.<n>Our large-scale experiments reveal that variations in state representation can lead to improvements greater than those achieved by changing the bandit algorithm itself.
• Score: 1.321203201549798
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: With the increasing availability of online information, recommender systems have become an important tool for many web-based systems. Due to the continuous aspect of recommendation environments, these systems increasingly rely on contextual multi-armed bandits (CMAB) to deliver personalized and real-time suggestions. A critical yet underexplored component in these systems is the representation of user state, which typically encapsulates the user's interaction history and is deeply correlated with the model's decisions and learning. In this paper, we investigate the impact of different embedding-based state representations derived from matrix factorization models on the performance of traditional CMAB algorithms. Our large-scale experiments reveal that variations in state representation can lead to improvements greater than those achieved by changing the bandit algorithm itself. Furthermore, no single embedding or aggregation strategy consistently dominates across datasets, underscoring the need for domain-specific evaluation. These results expose a substantial gap in the literature and emphasize that advancing bandit-based recommender systems requires a holistic approach that prioritizes embedding quality and state construction alongside algorithmic innovation. The source code for our experiments is publicly available on https://github.com/UFSCar-LaSID/bandits_blind_spot.

Related papers

• Multistakeholder Impacts of Profile Portability in a Recommender Ecosystem [0.7711381100220001]
  This paper explores the implications of algorithmic pluralism (also known as "middleware" in the governance literature) in which recommendation algorithms are decoupled from platforms.<n>We examine how data portability scenarios produce varying effects on user utility across different recommendation algorithms.
  arXiv  Detail & Related papers  (2026-04-23T14:59:54Z)
• Generalized Low-Rank Matrix Contextual Bandits with Graph Information [10.955203089942582]
  The matrix contextual bandit (CB) is a powerful framework that has been widely applied in sequential decision-making scenarios.<n>In many real-world scenarios, such as online advertising and recommender systems, additional graph information often exists beyond the low-rank structure.<n>We propose a novel matrix CB algorithmic framework that builds upon the classical upper confidence bound (UCB) framework.
  arXiv  Detail & Related papers  (2025-07-23T14:07:47Z)
• Epistemic Uncertainty-aware Recommendation Systems via Bayesian Deep Ensemble Learning [2.3310092106321365]
  We propose an ensemble-based supermodel to generate more robust and reliable predictions.<n>We also introduce a new interpretable non-linear matching approach for the user and item embeddings.
  arXiv  Detail & Related papers  (2025-04-14T23:04:35Z)
• Interactive Visualization Recommendation with Hier-SUCB [52.11209329270573]
  We propose an interactive personalized visualization recommendation (PVisRec) system that learns on user feedback from previous interactions.<n>For more interactive and accurate recommendations, we propose Hier-SUCB, a contextual semi-bandit in the PVisRec setting.
  arXiv  Detail & Related papers  (2025-02-05T17:14:45Z)
• Online Clustering of Dueling Bandits [59.09590979404303]
  We introduce the first "clustering of dueling bandit algorithms" to enable collaborative decision-making based on preference feedback.<n>We propose two novel algorithms: (1) Clustering of Linear Dueling Bandits (COLDB) which models the user reward functions as linear functions of the context vectors, and (2) Clustering of Neural Dueling Bandits (CONDB) which uses a neural network to model complex, non-linear user reward functions.
  arXiv  Detail & Related papers  (2025-02-04T07:55:41Z)
• Hierarchical Reinforcement Learning for Temporal Abstraction of Listwise Recommendation [51.06031200728449]
  We propose a novel framework called mccHRL to provide different levels of temporal abstraction on listwise recommendation.<n>Within the hierarchical framework, the high-level agent studies the evolution of user perception, while the low-level agent produces the item selection policy.<n>Results observe significant performance improvement by our method, compared with several well-known baselines.
  arXiv  Detail & Related papers  (2024-09-11T17:01:06Z)
• Pistis-RAG: Enhancing Retrieval-Augmented Generation with Human Feedback [41.88662700261036]
  RAG systems face limitations when semantic relevance alone does not guarantee improved generation quality. We propose Pistis-RAG, a new RAG framework designed with a content-centric approach to better align LLMs with human preferences.
  arXiv  Detail & Related papers  (2024-06-21T08:52:11Z)
• Embedding in Recommender Systems: A Survey [54.55152033023537]
  This survey presents a comprehensive analysis of advances in recommender system embedding techniques.<n>In matrix-based scenarios, collaborative filtering generates embeddings that effectively model user-item preferences.<n>We introduce emerging approaches, including AutoML, hashing techniques, and quantization methods, to enhance performance.
  arXiv  Detail & Related papers  (2023-10-28T06:31:06Z)
• Age of Semantics in Cooperative Communications: To Expedite Simulation Towards Real via Offline Reinforcement Learning [53.18060442931179]
  We propose the age of semantics (AoS) for measuring semantics freshness of status updates in a cooperative relay communication system. We derive an online deep actor-critic (DAC) learning scheme under the on-policy temporal difference learning framework. We then put forward a novel offline DAC scheme, which estimates the optimal control policy from a previously collected dataset.
  arXiv  Detail & Related papers  (2022-09-19T11:55:28Z)
• Ordinal Graph Gamma Belief Network for Social Recommender Systems [54.9487910312535]
  We develop a hierarchical Bayesian model termed ordinal graph factor analysis (OGFA), which jointly models user-item and user-user interactions. OGFA not only achieves good recommendation performance, but also extracts interpretable latent factors corresponding to representative user preferences. We extend OGFA to ordinal graph gamma belief network, which is a multi-stochastic-layer deep probabilistic model.
  arXiv  Detail & Related papers  (2022-09-12T09:19:22Z)
• Bayesian Non-stationary Linear Bandits for Large-Scale Recommender Systems [6.009759445555003]
  We build upon the linear contextual multi-armed bandit framework to address this problem. We develop a decision-making policy for a linear bandit problem with high-dimensional feature vectors. Our proposed recommender system employs this policy to learn the users' item preferences online while minimizing runtime.
  arXiv  Detail & Related papers  (2022-02-07T13:51:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.