A Unified Causal Framework for Auditing Recommender Systems for Ethical Concerns

• URL: http://arxiv.org/abs/2409.13210v1
• Date: Fri, 20 Sep 2024 04:37:36 GMT
• Title: A Unified Causal Framework for Auditing Recommender Systems for Ethical Concerns
• Authors: Vibhhu Sharma, Shantanu Gupta, Nil-Jana Akpinar, Zachary C. Lipton, Liu Leqi,
• Abstract summary: We view recommender system auditing from a causal lens and provide a general recipe for defining auditing metrics. Under this general causal auditing framework, we categorize existing auditing metrics and identify gaps in them. We propose two classes of such metrics:future- and past-reacheability and stability, that measure the ability of a user to influence their own and other users' recommendations.
• Score: 40.793466500324904
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As recommender systems become widely deployed in different domains, they increasingly influence their users' beliefs and preferences. Auditing recommender systems is crucial as it not only ensures the continuous improvement of recommendation algorithms but also safeguards against potential issues like biases and ethical concerns. In this paper, we view recommender system auditing from a causal lens and provide a general recipe for defining auditing metrics. Under this general causal auditing framework, we categorize existing auditing metrics and identify gaps in them -- notably, the lack of metrics for auditing user agency while accounting for the multi-step dynamics of the recommendation process. We leverage our framework and propose two classes of such metrics:future- and past-reacheability and stability, that measure the ability of a user to influence their own and other users' recommendations, respectively. We provide both a gradient-based and a black-box approach for computing these metrics, allowing the auditor to compute them under different levels of access to the recommender system. In our experiments, we demonstrate the efficacy of methods for computing the proposed metrics and inspect the design of recommender systems through these proposed metrics.

Related papers

• Quantifying User Coherence: A Unified Framework for Cross-Domain Recommendation Analysis [69.37718774071793]
  This paper introduces novel information-theoretic measures for understanding recommender systems. We evaluate 7 recommendation algorithms across 9 datasets, revealing the relationships between our measures and standard performance metrics.
  arXiv  Detail & Related papers  (2024-10-03T13:02:07Z)
• Revisiting Reciprocal Recommender Systems: Metrics, Formulation, and Method [60.364834418531366]
  We propose five new evaluation metrics that comprehensively and accurately assess the performance of RRS. We formulate the RRS from a causal perspective, formulating recommendations as bilateral interventions. We introduce a reranking strategy to maximize matching outcomes, as measured by the proposed metrics.
  arXiv  Detail & Related papers  (2024-08-19T07:21:02Z)
• A Comprehensive Survey of Evaluation Techniques for Recommendation Systems [0.0]
  This paper introduces a comprehensive suite of metrics, each tailored to capture a distinct aspect of system performance. We identify the strengths and limitations of current evaluation practices and highlight the nuanced trade-offs that emerge when optimizing recommendation systems across different metrics.
  arXiv  Detail & Related papers  (2023-12-26T11:57:01Z)
• RecRec: Algorithmic Recourse for Recommender Systems [41.97186998947909]
  It is crucial for all stakeholders to understand the model's rationale behind making certain predictions and recommendations. This is especially true for the content providers whose livelihoods depend on the recommender system. We propose a recourse framework for recommender systems, targeted towards the content providers.
  arXiv  Detail & Related papers  (2023-08-28T22:26:50Z)
• User-Controllable Recommendation via Counterfactual Retrospective and Prospective Explanations [96.45414741693119]
  We present a user-controllable recommender system that seamlessly integrates explainability and controllability. By providing both retrospective and prospective explanations through counterfactual reasoning, users can customize their control over the system.
  arXiv  Detail & Related papers  (2023-08-02T01:13:36Z)
• A Survey on Fairness-aware Recommender Systems [59.23208133653637]
  We present concepts of fairness in different recommendation scenarios, comprehensively categorize current advances, and introduce typical methods to promote fairness in different stages of recommender systems. Next, we delve into the significant influence that fairness-aware recommender systems exert on real-world industrial applications.
  arXiv  Detail & Related papers  (2023-06-01T07:08:22Z)
• Breaking Feedback Loops in Recommender Systems with Causal Inference [99.22185950608838]
  Recent work has shown that feedback loops may compromise recommendation quality and homogenize user behavior. We propose the Causal Adjustment for Feedback Loops (CAFL), an algorithm that provably breaks feedback loops using causal inference. We show that CAFL improves recommendation quality when compared to prior correction methods.
  arXiv  Detail & Related papers  (2022-07-04T17:58:39Z)
• Experiments on Generalizability of User-Oriented Fairness in Recommender Systems [2.0932879442844476]
  A fairness-aware recommender system aims to treat different user groups similarly. We propose a user-centered fairness re-ranking framework applied on top of a base ranking model. We evaluate the final recommendations provided by the re-ranking framework from both user- (e.g., NDCG) and item-side (e.g., novelty, item-fairness) metrics.
  arXiv  Detail & Related papers  (2022-05-17T12:36:30Z)
• Online certification of preference-based fairness for personalized recommender systems [20.875347023588652]
  We assess the fairness of personalized recommender systems in the sense of envy-freeness. We propose an auditing algorithm based on pure exploration and conservative constraints in multi-armed bandits.
  arXiv  Detail & Related papers  (2021-04-29T17:45:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.