ARTAI: An Evaluation Platform to Assess Societal Risk of Recommender Algorithms

• URL: http://arxiv.org/abs/2409.12396v1
• Date: Thu, 19 Sep 2024 01:39:51 GMT
• Title: ARTAI: An Evaluation Platform to Assess Societal Risk of Recommender Algorithms
• Authors: Qin Ruan, Jin Xu, Ruihai Dong, Arjumand Younus, Tai Tan Mai, Barry O'Sullivan, Susan Leavy,
• Abstract summary: ARTAI is an evaluation environment that enables large-scale assessments of recommender algorithms. This paper presents ARTAI, an evaluation environment that enables large-scale assessments of recommender algorithms to identify harmful patterns in how content is distributed online.
• Score: 6.697530342907843
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Societal risk emanating from how recommender algorithms disseminate content online is now well documented. Emergent regulation aims to mitigate this risk through ethical audits and enabling new research on the social impact of algorithms. However, there is currently a need for tools and methods that enable such evaluation. This paper presents ARTAI, an evaluation environment that enables large-scale assessments of recommender algorithms to identify harmful patterns in how content is distributed online and enables the implementation of new regulatory requirements for increased transparency in recommender systems.

Related papers

• An Overview of the Risk-based Model of AI Governance [0.0]
  The 'Analysis' section proposes several criticisms of the risk based approach to AI governance.<n>It argues that the notion of risk is problematic as its inherent normativity reproduces dominant and harmful narratives about whose interests matter.<n>This paper concludes with the suggestion that existing risk governance scholarship can provide valuable insights toward the improvement of the risk based AI governance.
  arXiv  Detail & Related papers  (2025-07-21T06:56:04Z)
• Planning and Learning in Average Risk-aware MDPs [4.696083734269232]
  We extend risk-neutral algorithms to accommodate the more general class of dynamic risk measures. Both the RVI and Q-learning algorithms are proven to converge to optimality. Our approach enables the identification of policies that are finely tuned to the intricate risk-awareness of the agent that they serve.
  arXiv  Detail & Related papers  (2025-03-22T03:18:09Z)
• Risk-Averse Certification of Bayesian Neural Networks [70.44969603471903]
  We propose a Risk-Averse Certification framework for Bayesian neural networks called RAC-BNN. Our method leverages sampling and optimisation to compute a sound approximation of the output set of a BNN. We validate RAC-BNN on a range of regression and classification benchmarks and compare its performance with a state-of-the-art method.
  arXiv  Detail & Related papers  (2024-11-29T14:22:51Z)
• Meta Clustering of Neural Bandits [45.77505279698894]
  We study a new problem, Clustering of Neural Bandits, by extending previous work to the arbitrary reward function. We propose a novel algorithm called M-CNB, which utilizes a meta-learner to represent and rapidly adapt to dynamic clusters. In extensive experiments conducted in both recommendation and online classification scenarios, M-CNB outperforms SOTA baselines.
  arXiv  Detail & Related papers  (2024-08-10T16:09:51Z)
• Evaluating Ensemble Methods for News Recommender Systems [50.90330146667386]
  This paper demonstrates how ensemble methods can be used to combine many diverse state-of-the-art algorithms to achieve superior results on the Microsoft News dataset (MIND) Our findings demonstrate that a combination of NRS algorithms can outperform individual algorithms, provided that the base learners are sufficiently diverse.
  arXiv  Detail & Related papers  (2024-06-23T13:40:50Z)
• Bayesian Safe Policy Learning with Chance Constrained Optimization: Application to Military Security Assessment during the Vietnam War [0.0]
  We investigate whether it would have been possible to improve a security assessment algorithm employed during the Vietnam War. This empirical application raises several methodological challenges that frequently arise in high-stakes algorithmic decision-making.
  arXiv  Detail & Related papers  (2023-07-17T20:59:50Z)
• Provably Efficient Iterated CVaR Reinforcement Learning with Function Approximation and Human Feedback [57.6775169085215]
  Risk-sensitive reinforcement learning aims to optimize policies that balance the expected reward and risk. We present a novel framework that employs an Iterated Conditional Value-at-Risk (CVaR) objective under both linear and general function approximations. We propose provably sample-efficient algorithms for this Iterated CVaR RL and provide rigorous theoretical analysis.
  arXiv  Detail & Related papers  (2023-07-06T08:14:54Z)
• Quantitative AI Risk Assessments: Opportunities and Challenges [9.262092738841979]
  AI-based systems are increasingly being leveraged to provide value to organizations, individuals, and society. Risks have led to proposed regulations, litigation, and general societal concerns. This paper explores the concept of a quantitative AI Risk Assessment.
  arXiv  Detail & Related papers  (2022-09-13T21:47:25Z)
• Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment [0.0]
  We develop a robust optimization approach that partially identifies the expected utility of a policy, and then finds an optimal policy. We extend this approach to common and important settings where humans make decisions with the aid of algorithmic recommendations. We derive new classification and recommendation rules that retain the transparency and interpretability of the existing risk assessment instrument.
  arXiv  Detail & Related papers  (2021-09-22T00:52:03Z)
• Safe Learning and Optimization Techniques: Towards a Survey of the State of the Art [3.6954802719347413]
  Safe learning and optimization deals with learning and optimization problems that avoid, as much as possible, the evaluation of non-safe input points. A comprehensive survey of safe reinforcement learning algorithms was published in 2015, but related works in active learning and in optimization were not considered. This paper reviews those algorithms from a number of domains including reinforcement learning, Gaussian process regression and classification, evolutionary algorithms, and active learning.
  arXiv  Detail & Related papers  (2021-01-23T13:58:09Z)
• Risk-Constrained Thompson Sampling for CVaR Bandits [82.47796318548306]
  We consider a popular risk measure in quantitative finance known as the Conditional Value at Risk (CVaR) We explore the performance of a Thompson Sampling-based algorithm CVaR-TS under this risk measure.
  arXiv  Detail & Related papers  (2020-11-16T15:53:22Z)
• Reinforcement Learning as Iterative and Amortised Inference [62.997667081978825]
  We use the control as inference framework to outline a novel classification scheme based on amortised and iterative inference. We show that taking this perspective allows us to identify parts of the algorithmic design space which have been relatively unexplored.
  arXiv  Detail & Related papers  (2020-06-13T16:10:03Z)
• SAMBA: Safe Model-Based & Active Reinforcement Learning [59.01424351231993]
  SAMBA is a framework for safe reinforcement learning that combines aspects from probabilistic modelling, information theory, and statistics. We evaluate our algorithm on a variety of safe dynamical system benchmarks involving both low and high-dimensional state representations. We provide intuition as to the effectiveness of the framework by a detailed analysis of our active metrics and safety constraints.
  arXiv  Detail & Related papers  (2020-06-12T10:40:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.