Audits Under Resource, Data, and Access Constraints: Scaling Laws For Less Discriminatory Alternatives

• URL: http://arxiv.org/abs/2509.05627v1
• Date: Sat, 06 Sep 2025 07:23:25 GMT
• Title: Audits Under Resource, Data, and Access Constraints: Scaling Laws For Less Discriminatory Alternatives
• Authors: Sarah H. Cen, Salil Goyal, Zaynah Javed, Ananya Karthik, Percy Liang, Daniel E. Ho,
• Abstract summary: We present a procedure enabling claimants to determine if an LDA exists, even when they have limited compute, data, information, and model access.<n>We provide a novel closed-form upper bound for the loss-fairness frontier (PF)<n>We show how the claimant can use it to fit a PF in the "low-resource regime," then extrapolate the PF that applies to the (large) model being contested.
• Score: 41.35437079064223
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AI audits play a critical role in AI accountability and safety. One branch of the law for which AI audits are particularly salient is anti-discrimination law. Several areas of anti-discrimination law implicate the "less discriminatory alternative" (LDA) requirement, in which a protocol (e.g., model) is defensible if no less discriminatory protocol that achieves comparable performance can be found with a reasonable amount of effort. Notably, the burden of proving an LDA exists typically falls on the claimant (the party alleging discrimination). This creates a significant hurdle in AI cases, as the claimant would seemingly need to train a less discriminatory yet high-performing model, a task requiring resources and expertise beyond most litigants. Moreover, developers often shield information about and access to their model and training data as trade secrets, making it difficult to reproduce a similar model from scratch. In this work, we present a procedure enabling claimants to determine if an LDA exists, even when they have limited compute, data, information, and model access. We focus on the setting in which fairness is given by demographic parity and performance by binary cross-entropy loss. As our main result, we provide a novel closed-form upper bound for the loss-fairness Pareto frontier (PF). We show how the claimant can use it to fit a PF in the "low-resource regime," then extrapolate the PF that applies to the (large) model being contested, all without training a single large model. The expression thus serves as a scaling law for loss-fairness PFs. To use this scaling law, the claimant would require a small subsample of the train/test data. Then, the claimant can fit the context-specific PF by training as few as 7 (small) models. We stress test our main result in simulations, finding that our scaling law holds even when the exact conditions of our theory do not.

Related papers

• Semantic Tube Prediction: Beating LLM Data Efficiency with JEPA [50.494504099850325]
  We introduce the Geodesic Hypothesis, positing that token sequences trace geodesics on a smooth semantic manifold and are therefore locally linear.<n>We show this constraint improves signal-to-noise ratio, and preserves diversity by preventing collisions during trajectory.<n>We demonstrate that geometric priors can surpass brute-force scaling.
  arXiv  Detail & Related papers  (2026-02-26T04:45:07Z)
• Statistical Guarantees in the Search for Less Discriminatory Algorithms [4.8750736477712815]
  We formalize LDA search via model multiplicity as an optimal stopping problem.<n>We provide a framework under which developers can impose stronger assumptions about the distribution of models.
  arXiv  Detail & Related papers  (2025-12-30T02:20:52Z)
• FairPFN: A Tabular Foundation Model for Causal Fairness [39.83807136585407]
  Causal fairness provides a transparent, human-in-the-loop framework to mitigate algorithmic discrimination.<n>We propose FairPFN, a model pre-trained on synthetic causal fairness data to identify and mitigate the causal effects of protected attributes in its predictions.
  arXiv  Detail & Related papers  (2025-06-08T09:15:45Z)
• Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data [73.04828796123581]
  Supervised fine-tuning (SFT) has become a crucial step for aligning pretrained large language models (LLMs)<n>We introduce Discriminative Fine-Tuning (DFT), an improved variant of SFT, which mitigates the burden of collecting human-labeled preference data.<n>Our contributions include: (i) a discriminative probabilistic framework for fine-tuning LLMs by explicitly modeling the discriminative likelihood of an answer among all possible outputs given an input; (ii) efficient algorithms to optimize this discriminative likelihood; and (iii) extensive experiments demonstrating DFT's effectiveness
  arXiv  Detail & Related papers  (2025-02-25T22:38:55Z)
• What Constitutes a Less Discriminatory Algorithm? [2.842548870013324]
  We argue that formal LDA definitions face fundamental challenges when they attempt to evaluate and compare predictive models in the absence of held-out data.<n>We put forward a framework in which both firms and plaintiffs can search for alternative models that comport with societal goals.
  arXiv  Detail & Related papers  (2024-12-24T03:49:48Z)
• The Legal Duty to Search for Less Discriminatory Algorithms [4.625678906362822]
  We argue that the law should place a duty of a reasonable search for LDAs. Model multiplicity and the availability of LDAs have significant ramifications for the legal response to discriminatory algorithms. We argue that the law should place a duty of a reasonable search for LDAs on entities that develop and deploy predictive models in covered civil rights domains.
  arXiv  Detail & Related papers  (2024-06-10T21:56:38Z)
• Fairness-Accuracy Trade-Offs: A Causal Perspective [58.06306331390586]
  We analyze the tension between fairness and accuracy from a causal lens for the first time.<n>We show that enforcing a causal constraint often reduces the disparity between demographic groups.<n>We introduce a new neural approach for causally-constrained fair learning.
  arXiv  Detail & Related papers  (2024-05-24T11:19:52Z)
• Fairness Without Harm: An Influence-Guided Active Sampling Approach [32.173195437797766]
  We aim to train models that mitigate group fairness disparity without causing harm to model accuracy. The current data acquisition methods, such as fair active learning approaches, typically require annotating sensitive attributes. We propose a tractable active data sampling algorithm that does not rely on training group annotations.
  arXiv  Detail & Related papers  (2024-02-20T07:57:38Z)
• Scaling Laws Beyond Backpropagation [64.0476282000118]
  We study the ability of Direct Feedback Alignment to train causal decoder-only Transformers efficiently. We find that DFA fails to offer more efficient scaling than backpropagation.
  arXiv  Detail & Related papers  (2022-10-26T10:09:14Z)
• Reusing the Task-specific Classifier as a Discriminator: Discriminator-free Adversarial Domain Adaptation [55.27563366506407]
  We introduce a discriminator-free adversarial learning network (DALN) for unsupervised domain adaptation (UDA) DALN achieves explicit domain alignment and category distinguishment through a unified objective. DALN compares favorably against the existing state-of-the-art (SOTA) methods on a variety of public datasets.
  arXiv  Detail & Related papers  (2022-04-08T04:40:18Z)
• Learning fair representation with a parametric integral probability metric [2.544539499281093]
  We propose a new adversarial training scheme for learning fair representation (LFR) In this paper, we derive theoretical relations between the fairness of representation and the fairness of the prediction model built on the top of the representation. Our proposed LFR algorithm is computationally lighter and more stable, and the final prediction model is competitive or superior to other LFR algorithms.
  arXiv  Detail & Related papers  (2022-02-07T05:02:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.