A Classification of Feedback Loops and Their Relation to Biases in Automated Decision-Making Systems

• URL: http://arxiv.org/abs/2305.06055v1
• Date: Wed, 10 May 2023 11:15:22 GMT
• Title: A Classification of Feedback Loops and Their Relation to Biases in Automated Decision-Making Systems
• Authors: Nicol\`o Pagan, Joachim Baumann, Ezzat Elokda, Giulia De Pasquale, Saverio Bolognani, Anik\'o Hann\'ak
• Abstract summary: We study the different types of feedback loops in the ML-based decision-making pipeline. We find that the existence of feedback loops can perpetuate, reinforce, or even reduce ML biases.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prediction-based decision-making systems are becoming increasingly prevalent in various domains. Previous studies have demonstrated that such systems are vulnerable to runaway feedback loops, e.g., when police are repeatedly sent back to the same neighborhoods regardless of the actual rate of criminal activity, which exacerbate existing biases. In practice, the automated decisions have dynamic feedback effects on the system itself that can perpetuate over time, making it difficult for short-sighted design choices to control the system's evolution. While researchers started proposing longer-term solutions to prevent adverse outcomes (such as bias towards certain groups), these interventions largely depend on ad hoc modeling assumptions and a rigorous theoretical understanding of the feedback dynamics in ML-based decision-making systems is currently missing. In this paper, we use the language of dynamical systems theory, a branch of applied mathematics that deals with the analysis of the interconnection of systems with dynamic behaviors, to rigorously classify the different types of feedback loops in the ML-based decision-making pipeline. By reviewing existing scholarly work, we show that this classification covers many examples discussed in the algorithmic fairness community, thereby providing a unifying and principled framework to study feedback loops. By qualitative analysis, and through a simulation example of recommender systems, we show which specific types of ML biases are affected by each type of feedback loop. We find that the existence of feedback loops in the ML-based decision-making pipeline can perpetuate, reinforce, or even reduce ML biases.

Related papers

• BlackBoxToBlueprint: Extracting Interpretable Logic from Legacy Systems using Reinforcement Learning and Counterfactual Analysis [0.0]
  This paper proposes a novel pipeline to automatically extract interpretable decision logic from legacy systems treated as black boxes.<n>The approach uses a Reinforcement Learning (RL) agent to explore the input space and identify critical decision boundaries by rewarding actions that cause meaningful changes in the system's output.<n>Decision trees are then trained on these clusters to extract human-readable rules that approximate the system's decision logic near the identified boundaries.
  arXiv  Detail & Related papers  (2025-06-30T18:36:54Z)
• Stream-Based Monitoring of Algorithmic Fairness [4.811789437743092]
  Stream-based monitoring is proposed as a solution for verifying the algorithmic fairness of decision and prediction systems at runtime. We present a principled way to formalize algorithmic fairness over temporal data streams in the specification language RTLola.
  arXiv  Detail & Related papers  (2025-01-30T13:18:59Z)
• Wait, that's not an option: LLMs Robustness with Incorrect Multiple-Choice Options [2.1184929769291294]
  This work introduces a novel framework for evaluating LLMs' capacity to balance instruction-following with critical reasoning.<n>We show that post-training aligned models often default to selecting invalid options, while base models exhibit improved refusal capabilities that scale with model size.<n>We additionally conduct a parallel human study showing similar instruction-following biases, with implications for how these biases may propagate through human feedback datasets used in alignment.
  arXiv  Detail & Related papers  (2024-08-27T19:27:43Z)
• Counterfactual-based Root Cause Analysis for Dynamical Systems [0.33748750222488655]
  We propose a causal method for root cause identification using a Residual Neural Network. We show that more root causes are identified when an intervention is performed on the structural equation and the external influence. We illustrate the effectiveness of the proposed method on a benchmark dynamic system as well as on a real world river dataset.
  arXiv  Detail & Related papers  (2024-06-12T11:38:13Z)
• A Mathematical Model of the Hidden Feedback Loop Effect in Machine Learning Systems [44.99833362998488]
  We introduce a repeated learning process to jointly describe several phenomena attributed to unintended hidden feedback loops. A distinctive feature of such repeated learning setting is that the state of the environment becomes causally dependent on the learner itself over time. We present a novel dynamical systems model of the repeated learning process and prove the limiting set of probability distributions for positive and negative feedback loop modes.
  arXiv  Detail & Related papers  (2024-05-04T17:57:24Z)
• Large-Scale Sequential Learning for Recommender and Engineering Systems [91.3755431537592]
  In this thesis, we focus on the design of an automatic algorithms that provide personalized ranking by adapting to the current conditions. For the former, we propose novel algorithm called SAROS that take into account both kinds of feedback for learning over the sequence of interactions. The proposed idea of taking into account the neighbour lines shows statistically significant results in comparison with the initial approach for faults detection in power grid.
  arXiv  Detail & Related papers  (2022-05-13T21:09:41Z)
• Investigation of Different Calibration Methods for Deep Speaker Embedding based Verification Systems [66.61691401921296]
  This paper presents an investigation over several methods of score calibration for deep speaker embedding extractors. An additional focus of this research is to estimate the impact of score normalization on the calibration performance of the system.
  arXiv  Detail & Related papers  (2022-03-28T21:22:22Z)
• Learning Physical Concepts in Cyber-Physical Systems: A Case Study [72.74318982275052]
  We provide an overview of the current state of research regarding methods for learning physical concepts in time series data. We also analyze the most important methods from the current state of the art using the example of a three-tank system.
  arXiv  Detail & Related papers  (2021-11-28T14:24:52Z)
• Stateful Offline Contextual Policy Evaluation and Learning [88.9134799076718]
  We study off-policy evaluation and learning from sequential data. We formalize the relevant causal structure of problems such as dynamic personalized pricing. We show improved out-of-sample policy performance in this class of relevant problems.
  arXiv  Detail & Related papers  (2021-10-19T16:15:56Z)
• Supervised DKRC with Images for Offline System Identification [77.34726150561087]
  Modern dynamical systems are becoming increasingly non-linear and complex. There is a need for a framework to model these systems in a compact and comprehensive representation for prediction and control. Our approach learns these basis functions using a supervised learning approach.
  arXiv  Detail & Related papers  (2021-09-06T04:39:06Z)
• Monitoring nonstationary processes based on recursive cointegration analysis and elastic weight consolidation [2.8102838347038617]
  Traditional approaches misidentify the normal dynamic deviations as faults and thus lead to high false alarms. In this paper, RCA and RPCA are proposed to distinguish the real faults from normal systems changes. elastic weight consolidation (EWC) is employed to settle the catastrophic forgetting' issue.
  arXiv  Detail & Related papers  (2021-01-21T12:49:18Z)
• Analysis of hidden feedback loops in continuous machine learning systems [0.0]
  We signify a problem of implicit feedback loops, demonstrate how they intervene with user behavior on an exemplary housing prices prediction system. Based on a preliminary model, we highlight conditions when such feedback loops arise and discuss possible solution approaches.
  arXiv  Detail & Related papers  (2021-01-14T15:43:18Z)
• An Uncertainty-based Human-in-the-loop System for Industrial Tool Wear Analysis [68.8204255655161]
  We show that uncertainty measures based on Monte-Carlo dropout in the context of a human-in-the-loop system increase the system's transparency and performance. A simulation study demonstrates that the uncertainty-based human-in-the-loop system increases performance for different levels of human involvement.
  arXiv  Detail & Related papers  (2020-07-14T15:47:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.