Bridging Prediction and Intervention Problems in Social Systems

• URL: http://arxiv.org/abs/2507.05216v1
• Date: Mon, 07 Jul 2025 17:29:13 GMT
• Title: Bridging Prediction and Intervention Problems in Social Systems
• Authors: Lydia T. Liu, Inioluwa Deborah Raji, Angela Zhou, Luke Guerdan, Jessica Hullman, Daniel Malinsky, Bryan Wilder, Simone Zhang, Hammaad Adam, Amanda Coston, Ben Laufer, Ezinne Nwankwo, Michael Zanger-Tishler, Eli Ben-Michael, Solon Barocas, Avi Feller, Marissa Gerchick, Talia Gillis, Shion Guha, Daniel Ho, Lily Hu, Kosuke Imai, Sayash Kapoor, Joshua Loftus, Razieh Nabi, Arvind Narayanan, Ben Recht, Juan Carlos Perdomo, Matthew Salganik, Mark Sendak, Alexander Tolbert, Berk Ustun, Suresh Venkatasubramanian, Angelina Wang, Ashia Wilson,
• Abstract summary: We consider the ways in which we must shift from a prediction-focused paradigm to an interventionist paradigm when considering the impact of automated decision systems (ADS)<n>We argue this requires a new default problem setup for ADS beyond prediction, to instead consider predictions as decision support, final decisions, and outcomes.<n>We highlight how this perspective unifies modern statistical frameworks and other tools to study the design, implementation, and evaluation of ADS systems.
• Score: 71.41519966787386
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many automated decision systems (ADS) are designed to solve prediction problems -- where the goal is to learn patterns from a sample of the population and apply them to individuals from the same population. In reality, these prediction systems operationalize holistic policy interventions in deployment. Once deployed, ADS can shape impacted population outcomes through an effective policy change in how decision-makers operate, while also being defined by past and present interactions between stakeholders and the limitations of existing organizational, as well as societal, infrastructure and context. In this work, we consider the ways in which we must shift from a prediction-focused paradigm to an interventionist paradigm when considering the impact of ADS within social systems. We argue this requires a new default problem setup for ADS beyond prediction, to instead consider predictions as decision support, final decisions, and outcomes. We highlight how this perspective unifies modern statistical frameworks and other tools to study the design, implementation, and evaluation of ADS systems, and point to the research directions necessary to operationalize this paradigm shift. Using these tools, we characterize the limitations of focusing on isolated prediction tasks, and lay the foundation for a more intervention-oriented approach to developing and deploying ADS.

Related papers

• Beyond Black-Box Benchmarking: Observability, Analytics, and Optimization of Agentic Systems [1.415098516077151]
  The rise of agentic AI systems, where agents collaborate to perform diverse tasks, poses new challenges with observing, analyzing and optimizing their behavior.<n>Traditional evaluation and benchmarking approaches struggle to handle the non-deterministic, context-sensitive, and dynamic nature of these systems.<n>This paper explores key challenges and opportunities in analyzing and optimizing agentic systems across development, testing, and maintenance.
  arXiv  Detail & Related papers  (2025-03-09T20:02:04Z)
• Evaluating Prediction-based Interventions with Human Decision Makers In Mind [1.192656186481075]
  We formalize and investigate various models of human decision-making in the presence of a predictive model aid.<n>We show that each of these behavioural models produces dependencies across decision subjects and results in the violation of existing assumptions.
  arXiv  Detail & Related papers  (2025-02-12T20:35:52Z)
• Trustworthy human-centric based Automated Decision-Making Systems [0.7048747239308888]
  Automated Decision-Making Systems (ADS) have become pervasive across various fields, activities, and occupations, to enhance performance. This research paper presents a thorough examination of the implications, distinctions, and ethical considerations associated with digitalization, digital transformation, and the utilization of ADS in contemporary society and future contexts.
  arXiv  Detail & Related papers  (2023-12-22T11:02:57Z)
• Bridging the gap: Towards an Expanded Toolkit for AI-driven Decision-Making in the Public Sector [6.693502127460251]
  AI-driven decision-making systems are becoming instrumental in the public sector, with applications spanning areas like criminal justice, social welfare, financial fraud detection, and public health. These systems face the challenge of aligning machine learning (ML) models with the complex realities of public sector decision-making. We examine five key challenges where misalignment can occur, including distribution shifts, label bias, the influence of past decision-making on the data side, as well as competing objectives and human-in-the-loop on the model output side.
  arXiv  Detail & Related papers  (2023-10-29T17:44:48Z)
• Predictable Artificial Intelligence [77.1127726638209]
  This paper introduces the ideas and challenges of Predictable AI.<n>It explores the ways in which we can anticipate key validity indicators of present and future AI ecosystems.<n>We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI ecosystems.
  arXiv  Detail & Related papers  (2023-10-09T21:36:21Z)
• A Closer Look at the Intervention Procedure of Concept Bottleneck Models [18.222350428973343]
  Concept bottleneck models (CBMs) are a class of interpretable neural network models that predict the target response of a given input based on its high-level concepts. CBMs enable domain experts to intervene on the predicted concepts and rectify any mistakes at test time, so that more accurate task predictions can be made at the end. We develop various ways of selecting intervening concepts to improve the intervention effectiveness and conduct an array of in-depth analyses as to how they evolve under different circumstances.
  arXiv  Detail & Related papers  (2023-02-28T02:37:24Z)
• Causal Fairness Analysis [68.12191782657437]
  We introduce a framework for understanding, modeling, and possibly solving issues of fairness in decision-making settings. The main insight of our approach will be to link the quantification of the disparities present on the observed data with the underlying, and often unobserved, collection of causal mechanisms. Our effort culminates in the Fairness Map, which is the first systematic attempt to organize and explain the relationship between different criteria found in the literature.
  arXiv  Detail & Related papers  (2022-07-23T01:06:34Z)
• Boosting the interpretability of clinical risk scores with intervention predictions [59.22442473992704]
  We propose a joint model of intervention policy and adverse event risk as a means to explicitly communicate the model's assumptions about future interventions. We show how combining typical risk scores, such as the likelihood of mortality, with future intervention probability scores leads to more interpretable clinical predictions.
  arXiv  Detail & Related papers  (2022-07-06T19:49:42Z)
• End-to-End Learning and Intervention in Games [60.41921763076017]
  We provide a unified framework for learning and intervention in games. We propose two approaches, respectively based on explicit and implicit differentiation. The analytical results are validated using several real-world problems.
  arXiv  Detail & Related papers  (2020-10-26T18:39:32Z)
• The Visual Social Distancing Problem [99.69094590087408]
  We introduce the Visual Social Distancing problem, defined as the automatic estimation of the inter-personal distance from an image. We discuss how VSD relates with previous literature in Social Signal Processing and indicate which existing Computer Vision methods can be used to manage such problem.
  arXiv  Detail & Related papers  (2020-05-11T00:04:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.