Deployment of AI-Assisted Interventions: Capacity Constraints and Noisy Compliance

• URL: http://arxiv.org/abs/2604.14370v1
• Date: Wed, 15 Apr 2026 19:40:35 GMT
• Title: Deployment of AI-Assisted Interventions: Capacity Constraints and Noisy Compliance
• Abstract summary: We show that policies based solely on predictive accuracy are generally suboptimal.<n>We introduce a new metric--Operational AUC--and show it leads to optimal algorithm selection.
• Score: 1.535642649272052
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI tools increasingly guide targeted interventions in healthcare, education, and recruiting. Algorithms score individuals, trigger outreach to those above a threshold (e.g., high-risk or high-value), and encourage them to request service; then providers deliver service to those who request. Standard practice sets the threshold and selects the algorithm to maximize predictive accuracy, assuming that better predictions yield better outcomes. We show that this approach is suboptimal when limited service capacity and probabilistic behavioral responses influence who receives service. In such settings, the optimal score threshold must balance two effects: ensuring all capacity is filled (utilization) and ensuring high-value individuals are served despite competition between requests (cannibalization). We characterize the optimal threshold and prove that policies based solely on predictive accuracy are generally suboptimal. Further, because optimal thresholds vary with service capacity, algorithm selection metrics like AUC, which weight all thresholds equally, are misaligned with operational performance. We introduce a new metric--Operational AUC (OpAUC)--and show it leads to optimal algorithm selection. Finally, we conduct a case study on sepsis early warning data and illustrate the magnitude of improvement that can be achieved from improved threshold and algorithm selection.

Related papers

• Aligning Data-Driven Predictors with Allocation: A Decision-Focused Approach to Survival Analysis [52.10284705210197]
  We show that machine learning predictors can yield arbitrarily poor outcomes when used for allocation.<n>We introduce a decision-focused learning approach based on optimizing normalized discounted cumulative gain (NDCG)<n>We propose a bootstrapping approach to optimize the NDCG of existing survival models.
  arXiv  Detail & Related papers  (2026-06-01T12:03:22Z)
• Resource-efficient quantum approximate optimization algorithm via Bayesian optimization and maximum-probability evaluation [0.15625709395072904]
  We present a resource-efficient QAOA framework that uses the cut value of the most probable measured bitstring as the optimization objective.<n>We show that, for both unweighted and weighted instances, the proposed scheme achieves discrete-solution quality comparable to that of the conventional expectation-based objective.
  arXiv  Detail & Related papers  (2026-03-30T13:22:25Z)
• Goal-Oriented Influence-Maximizing Data Acquisition for Learning and Optimization [28.53710231018475]
  We propose an active acquisition algorithm that avoids explicit posterior inference while remaining uncertainty-aware through inverse curvature.<n>GOIMDA selects inputs by maximizing their expected influence on a user-specified goal functional.<n>We show theoretically that, for generalized linear models, GOIMDA approximates predictive-entropy minimization up to a correction term accounting for goal alignment and prediction bias.
  arXiv  Detail & Related papers  (2026-02-23T07:57:11Z)
• Principled Algorithms for Optimizing Generalized Metrics in Binary Classification [53.604375124674796]
  We introduce principled algorithms for optimizing generalized metrics, supported by $H$-consistency and finite-sample generalization bounds.<n>Our approach reformulates metric optimization as a generalized cost-sensitive learning problem.<n>We develop new algorithms, METRO, with strong theoretical performance guarantees.
  arXiv  Detail & Related papers  (2025-12-29T01:33:42Z)
• On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows [71.92083784393418]
  Agentic AI (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low.<n>Inference-time alignment relies on three components: sampling, evaluation, and feedback.<n>We introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques.
  arXiv  Detail & Related papers  (2025-04-02T17:40:47Z)
• Understanding Likelihood Over-optimisation in Direct Alignment Algorithms [20.043560907227018]
  Direct Alignment Algorithms (DAAs) have emerged as alternatives to online Reinforcement Learning from Human Feedback. These algorithms aim to increase the likelihood of generating better (preferred) completions while discouraging worse (non-preferred) ones. This work explores the relationship between completion likelihood and model performance in state-of-the-art DAAs.
  arXiv  Detail & Related papers  (2024-10-15T15:14:22Z)
• Online Sensitivity Optimization in Differentially Private Learning [8.12606646175019]
  We present a novel approach to dynamically optimize the clipping threshold. We treat this threshold as an additional learnable parameter, establishing a clean relationship between the threshold and the cost function. Our method is thoroughly assessed against alternative fixed and adaptive strategies across diverse datasets, tasks, model dimensions, and privacy levels.
  arXiv  Detail & Related papers  (2023-10-02T00:30:49Z)
• Meta-Wrapper: Differentiable Wrapping Operator for User Interest Selection in CTR Prediction [97.99938802797377]
  Click-through rate (CTR) prediction, whose goal is to predict the probability of the user to click on an item, has become increasingly significant in recommender systems. Recent deep learning models with the ability to automatically extract the user interest from his/her behaviors have achieved great success. We propose a novel approach under the framework of the wrapper method, which is named Meta-Wrapper.
  arXiv  Detail & Related papers  (2022-06-28T03:28:15Z)
• Efficient and Differentiable Conformal Prediction with General Function Classes [96.74055810115456]
  We propose a generalization of conformal prediction to multiple learnable parameters. We show that it achieves approximate valid population coverage and near-optimal efficiency within class. Experiments show that our algorithm is able to learn valid prediction sets and improve the efficiency significantly.
  arXiv  Detail & Related papers  (2022-02-22T18:37:23Z)
• On the Optimality of Batch Policy Optimization Algorithms [106.89498352537682]
  Batch policy optimization considers leveraging existing data for policy construction before interacting with an environment. We show that any confidence-adjusted index algorithm is minimax optimal, whether it be optimistic, pessimistic or neutral. We introduce a new weighted-minimax criterion that considers the inherent difficulty of optimal value prediction.
  arXiv  Detail & Related papers  (2021-04-06T05:23:20Z)
• Double Coverage with Machine-Learned Advice [100.23487145400833]
  We study the fundamental online $k$-server problem in a learning-augmented setting. We show that our algorithm achieves for any k an almost optimal consistency-robustness tradeoff.
  arXiv  Detail & Related papers  (2021-03-02T11:04:33Z)
• Towards Feature-Based Performance Regression Using Trajectory Data [0.9281671380673306]
  Black-box optimization is a very active area of research, with many new algorithms being developed every year. The variety of algorithms poses a meta-problem: which algorithm to choose for a given problem at hand? Past research has shown that per-instance algorithm selection based on exploratory landscape analysis can be an efficient mean to tackle this meta-problem.
  arXiv  Detail & Related papers  (2021-02-10T10:19:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.