Can We Steer the Black-Box? Towards Controllability-Centric Evaluation of Recommender Systems with Collaborative Agents

• URL: http://arxiv.org/abs/2607.13418v2
• Date: Thu, 16 Jul 2026 00:42:41 GMT
• Title: Can We Steer the Black-Box? Towards Controllability-Centric Evaluation of Recommender Systems with Collaborative Agents
• Abstract summary: We propose CtrlBench-Rec, a framework for systematic assessment of controllability.<n>We formalize three fundamental tasks: target content discovery, interest profile shaping, and popularity bias mitigation.<n>Extensive experiments on real-world datasets and multiple recommendation models demonstrate that our framework effectively quantifies controllability.
• Score: 27.581560179325816
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recommender systems operate as Black-Boxes, leaving users and regulators unable to steer their outputs toward specific intentions or audit their behavior. This lack of controllability, defined as the system's ability to respond to explicit guidance, remains an unaddressed dimension in existing evaluation paradigms. To fill this gap, we propose CtrlBench-Rec, a collaborative multi-agent framework for systematic assessment of controllability. We formalize three fundamental tasks: target content discovery, interest profile shaping, and popularity bias mitigation, which together measure steerability from explicit commands to implicit representation steering and finally to overcoming algorithmic biases.Extensive experiments on real-world datasets and multiple recommendation models demonstrate that our framework effectively quantifies controllability and exposes critical system bottlenecks, most notably persistent resistance to guiding long tail content. CtrlBench-Rec provides the first standardized toolkit for controllable recommendation research, algorithmic auditing, and user empowerment. Our code is released on https://github.com/caskcsg/CtrlBenchRec.

Related papers

• Verifiable Reasoning for LLM-based Generative Recommendation [106.7765000777685]
  Reasoning in Large Language Models (LLMs) has recently shown strong potential in enhancing generative recommendation.<n>We propose a novel textbftextitreason-verify-recommend paradigm, which interleaves reasoning with verification to provide reliable feedback.<n>Experiments on four real-world datasets demonstrate that VRec substantially enhances recommendation effectiveness and scalability without compromising efficiency.
  arXiv  Detail & Related papers  (2026-03-08T16:55:45Z)
• Token-Controlled Re-ranking for Sequential Recommendation via LLMs [32.61510378078676]
  COREC is a novel token-augmented re-ranking framework that incorporates specific user requirements in co-creating the recommendation outcome.<n> COREC empowers users to steer re-ranking results with precise and flexible control via explicit, attribute-based signals.<n> Experiments show that COREC exceeds state-of-the-art baselines on standard recommendation effectiveness.
  arXiv  Detail & Related papers  (2025-11-22T04:31:19Z)
• Robust Verification of Controllers under State Uncertainty via Hamilton-Jacobi Reachability Analysis [49.31947916567367]
  Hamilton-Jacobi (J) reachability analysis is a popular formal verification tool for general nonlinear systems that can compute optimal reachable under worst-case uncertainties.<n>This work is the first HJ-based reachability-based system verification framework for the Robust Verification Controllers via HJ rover.<n>Within Ro-CoRe, we propose novel methods for safety verification and controller design.
  arXiv  Detail & Related papers  (2025-11-18T18:55:20Z)
• A Unified Causal Framework for Auditing Recommender Systems for Ethical Concerns [40.793466500324904]
  We view recommender system auditing from a causal lens and provide a general recipe for defining auditing metrics. Under this general causal auditing framework, we categorize existing auditing metrics and identify gaps in them. We propose two classes of such metrics:future- and past-reacheability and stability, that measure the ability of a user to influence their own and other users' recommendations.
  arXiv  Detail & Related papers  (2024-09-20T04:37:36Z)
• An Efficient Continuous Control Perspective for Reinforcement-Learning-based Sequential Recommendation [14.506332665769746]
  We propose an underlinetextbfEfficient underlinetextbfContinuous underlinetextbfControl framework (ECoC) Based on a statistically tested assumption, we first propose the novel unified action representation abstracted from normalized user and item spaces. During this process, strategic exploration and directional control in terms of unified actions are carefully designed and crucial to final recommendation decisions.
  arXiv  Detail & Related papers  (2024-08-15T09:26:26Z)
• User-Controllable Recommendation via Counterfactual Retrospective and Prospective Explanations [96.45414741693119]
  We present a user-controllable recommender system that seamlessly integrates explainability and controllability. By providing both retrospective and prospective explanations through counterfactual reasoning, users can customize their control over the system.
  arXiv  Detail & Related papers  (2023-08-02T01:13:36Z)
• Recommendation Systems with Distribution-Free Reliability Guarantees [83.80644194980042]
  We show how to return a set of items rigorously guaranteed to contain mostly good items. Our procedure endows any ranking model with rigorous finite-sample control of the false discovery rate. We evaluate our methods on the Yahoo! Learning to Rank and MSMarco datasets.
  arXiv  Detail & Related papers  (2022-07-04T17:49:25Z)
• Controllable Summarization with Constrained Markov Decision Process [50.04321779376415]
  We study controllable text summarization which allows users to gain control on a particular attribute. We propose a novel training framework based on Constrained Markov Decision Process (CMDP) Our framework can be applied to control important attributes of summarization, including length, covered entities, and abstractiveness.
  arXiv  Detail & Related papers  (2021-08-07T09:12:53Z)
• Controllable Multi-Interest Framework for Recommendation [64.30030600415654]
  We formalize the recommender system as a sequential recommendation problem. We propose a novel controllable multi-interest framework for the sequential recommendation, called ComiRec. Our framework has been successfully deployed on the offline Alibaba distributed cloud platform.
  arXiv  Detail & Related papers  (2020-05-19T10:18:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.