Anterior's Approach to Fairness Evaluation of Automated Prior Authorization System

• URL: http://arxiv.org/abs/2603.14631v1
• Date: Sun, 15 Mar 2026 22:05:24 GMT
• Title: Anterior's Approach to Fairness Evaluation of Automated Prior Authorization System
• Abstract summary: We propose a fairness evaluation framework for prior authorization models based on model error rates rather than approval outcomes.<n>Using 7,166 human-reviewed cases spanning 27 medical necessity guidelines, we assessed consistency in sex, age, race/ethnicity, and socioeconomic status.
• Score: 0.10195618602298682
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Increasing staffing constraints and turnaround-time pressures in Prior authorization (PA) have led to increasing automation of decision systems to support PA review. Evaluating fairness in such systems poses unique challenges because legitimate clinical guidelines and medical necessity criteria often differ across demographic groups, making parity in approval rates an inappropriate fairness metric. We propose a fairness evaluation framework for prior authorization models based on model error rates rather than approval outcomes. Using 7,166 human-reviewed cases spanning 27 medical necessity guidelines, we assessed consistency in sex, age, race/ethnicity, and socioeconomic status. Our evaluation combined error-rate comparisons, tolerance-band analysis with a predefined $\pm$5 percentage-point margin, statistical power evaluation, and protocol-controlled logistic regression. Across most demographics, model error rates were consistent, and confidence intervals fell within the predefined tolerance band, indicating no meaningful performance differences. For race/ethnicity, point estimates remain small, but subgroup sample sizes were limited, resulting in wide confidence intervals and underpowered tests, with inconclusive evidence within the dataset we explored. These findings illustrate a rigorous and regulator-aligned approach to fairness evaluation in administrative healthcare AI systems.

Related papers

• To Compare, or Not to Compare: On Methodological Practices in Evaluating Social Bias [61.40435736418359]
  We introduce a unified framework that standardizes heterogeneous benchmarks to contrast isolated demographic assessments with forced-choice comparative settings.<n>We show that comparative settings act as aggressive catalysts for latent discrimination, a shift primarily driven by underspecified contexts.<n>Finally, we demonstrate that this comparative prejudice is a generalized phenomenon that scales positively with model size.
  arXiv  Detail & Related papers  (2026-06-23T13:53:50Z)
• RISED: A Pre-Deployment Safety Evaluation Framework for Clinical AI Decision-Support Systems [0.0]
  We propose a five-dimension pre-deployment evaluation covering Reliability, Inclusivity, Sensitivity, Equity, and Deployability.<n> RISED is released as an open-source Python package that supplies the quantitative verdicts existing clinical AI reporting standards require.
  arXiv  Detail & Related papers  (2026-05-13T02:17:13Z)
• When Fairness Metrics Disagree: Evaluating the Reliability of Demographic Fairness Assessment in Machine Learning [0.0]
  We investigate the consistency of fairness evaluation by conducting a systematic multi-metric analysis of demographic bias in machine learning models.<n>Our results demonstrate that fairness assessments can vary significantly depending on the choice of metrics, leading to contradictory conclusions regarding model bias.<n>These findings highlight a critical limitation in current fairness evaluation practices and suggest that single-metric reporting is insufficient for reliable bias assessment.
  arXiv  Detail & Related papers  (2026-04-16T14:07:37Z)
• Empirical Likelihood-Based Fairness Auditing: Distribution-Free Certification and Flagging [18.71249153088185]
  Machine learning models in high-stakes applications, such as recidivism prediction and automated personnel selection, often exhibit systematic performance disparities.<n>We propose a novel empirical likelihood-based (EL) framework that constructs robust statistical measures for model performance disparities.
  arXiv  Detail & Related papers  (2026-01-28T05:36:19Z)
• Understanding challenges to the interpretation of disaggregated evaluations of algorithmic fairness [49.35494016290887]
  We show that equal performance across subgroups is an unreliable measure of fairness when data are representative of relevant populations but reflective of real-world disparities.<n>Our framework suggests complementing disaggregated evaluations with explicit causal assumptions and analysis to control for confounding and distribution shift.
  arXiv  Detail & Related papers  (2025-06-04T17:40:31Z)
• Fairness Perceptions in Regression-based Predictive Models [0.19999259391104385]
  This paper introduces three novel divergence-based group fairness notions to assess the fairness of regression-based analytics tools.<n>Crowd feedback is used to estimate social fairness preferences.<n>The findings clearly depict a strong preference towards the separation and sufficiency fairness notions, and that the predictive analytics is deemed fair with respect to gender and race groups, but unfair in terms of age groups.
  arXiv  Detail & Related papers  (2025-05-08T01:48:27Z)
• Conformal uncertainty quantification to evaluate predictive fairness of foundation AI model for skin lesion classes across patient demographics [8.692647930497936]
  We use conformal analysis to quantify the predictive uncertainty of a vision transformer based foundation model.<n>We show how this can be used as a fairness metric to evaluate the robustness of the feature embeddings of the foundation model.
  arXiv  Detail & Related papers  (2025-03-31T08:06:00Z)
• Position: AI Evaluation Should Learn from How We Test Humans [65.36614996495983]
  We argue that psychometrics, a theory originating in the 20th century for human assessment, could be a powerful solution to the challenges in today's AI evaluations.
  arXiv  Detail & Related papers  (2023-06-18T09:54:33Z)
• Error Parity Fairness: Testing for Group Fairness in Regression Tasks [5.076419064097733]
  This work presents error parity as a regression fairness notion and introduces a testing methodology to assess group fairness. It is followed by a suitable permutation test to compare groups on several statistics to explore disparities and identify impacted groups. Overall, the proposed regression fairness testing methodology fills a gap in the fair machine learning literature and may serve as a part of larger accountability assessments and algorithm audits.
  arXiv  Detail & Related papers  (2022-08-16T17:47:20Z)
• Measuring Fairness of Text Classifiers via Prediction Sensitivity [63.56554964580627]
  ACCUMULATED PREDICTION SENSITIVITY measures fairness in machine learning models based on the model's prediction sensitivity to perturbations in input features. We show that the metric can be theoretically linked with a specific notion of group fairness (statistical parity) and individual fairness.
  arXiv  Detail & Related papers  (2022-03-16T15:00:33Z)
• Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach [131.20444904674494]
  We tackle the problem of measuring group fairness under unawareness of sensitive attributes. We show that quantification approaches are particularly suited to tackle the fairness-under-unawareness problem.
  arXiv  Detail & Related papers  (2021-09-17T13:45:46Z)
• Estimating and Improving Fairness with Adversarial Learning [65.99330614802388]
  We propose an adversarial multi-task training strategy to simultaneously mitigate and detect bias in the deep learning-based medical image analysis system. Specifically, we propose to add a discrimination module against bias and a critical module that predicts unfairness within the base classification model. We evaluate our framework on a large-scale public-available skin lesion dataset.
  arXiv  Detail & Related papers  (2021-03-07T03:10:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.