Related papers: Who Gets Heeded? An Obligation-Level Audit of Responsiveness in EPA Rulemaking

Who Gets Heeded? An Obligation-Level Audit of Responsiveness in EPA Rulemaking

URL: http://arxiv.org/abs/2608.10329v1
Date: Tue, 11 Aug 2026 00:05:58 GMT
Title: Who Gets Heeded? An Obligation-Level Audit of Responsiveness in EPA Rulemaking
Abstract summary: We introduce obligation-level responsiveness auditing, an auditable, AI-assisted framework for measuring whether public-comment engagement co-occurs with changes to specific regulatory duties.<n>We apply the framework to 70,075 comments across 36 EPA anchor rulemakings, drawn from a corpus of 786,197 comments across 6,145 dockets from 2010-2022.
Score: 10.141053299773306
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Notice-and-comment rulemaking gives any affected party the same formal right to influence federal regulation, but formal access is not substantive capacity to shape rule text. Existing strategies operate at the rule or aggregate-corpus level, too coarse to capture the discrete regulatory obligations where commenters seek change. We introduce obligation-level responsiveness auditing, an auditable, AI-assisted framework for measuring whether public-comment engagement co-occurs with changes to specific regulatory duties. The framework extracts proposed and final-rule obligations, matches comments to the obligations they address, and classifies proposed-final outcomes; each load-bearing component is evaluated against blind human judgment. We apply the framework to 70,075 comments across 36 EPA anchor rulemakings, drawn from a corpus of 786,197 comments across 6,145 dockets from 2010-2022. Three descriptive findings emerge. First, engagement is associated with revision at a modest within-docket magnitude. Second, support-versus-opposition direction does not clearly differentiate outcomes, an informative null inconsistent with simple preference-aggregation. Third, under a permissive reconstruction of commenter type, organizational-majority engagement concentrates in editorial-refinement rather than substantive-modification outcomes at the cross-docket level. A blind human audit of the load-bearing outcome contrast preserves this third finding under corrected labels and reveals that text-similarity methods are insufficient for distinguishing editorial from substantive regulatory change, a measurement-validity lesson we treat as a supporting methodological contribution. Together, these findings locate the equity asymmetry upstream of agency response: in differential capacity across commenter populations to identify, interpret, and contest specific legal obligations.

Related papers

Who judges the judges? Governance from metrics: a runtime framework for continuous LLM compliance monitoring [0.0]
Current approaches to AI compliance treat conformity as a binary, audit-time verdict.<n>We argue that this compliance fiction is structurally ill-suited to the requirements of the EU AI Act.<n>We introduce governance from metrics, a principle whereby regulatory compliance is derived as a continuous signal from behavioural observability.
arXiv Detail & Related papers (2026-05-23T21:21:33Z)
Quantifying the Statistical Effect of Rubric Modifications on Human-Autorater Agreement [50.34437999083224]
Autoraters are increasingly used for evaluation and automated moderation content.<n>There is limited statistical analysis of how modifications in a rubric presented to both humans and autoraters affect their score agreement.
arXiv Detail & Related papers (2026-05-07T13:55:11Z)
Escaping the Agreement Trap: Defensibility Signals for Evaluating Rule-Governed AI [0.6138671548064355]
We formalize evaluation as policy-grounded correctness and introduce the Defensibility Index (DI) and Ambiguity Index (AI)<n>We validate the framework on 193,000+ Reddit moderation decisions across multiple communities and evaluation cohorts.
arXiv Detail & Related papers (2026-04-22T18:05:29Z)
De Jure: Iterative LLM Self-Refinement for Structured Extraction of Regulatory Rules [1.6865681749065902]
De Jure is a fully automated, domain-agnostic pipeline for extracting structured regulatory rules from raw documents.<n>We evaluate De Jure across four models on three regulatory corpora spanning finance, healthcare, and AI governance.
arXiv Detail & Related papers (2026-04-02T17:06:50Z)
Preventing the Collapse of Peer Review Requires Verification-First AI [49.995126139461085]
We propose truth-coupling, i.e. how tightly venue scores track latent scientific truth.<n>We formalize two forces that drive a phase transition toward proxy-sovereign evaluation.
arXiv Detail & Related papers (2026-01-23T17:17:32Z)
Judging by the Rules: Compliance-Aligned Framework for Modern Slavery Statement Monitoring [24.13989765643719]
Modern slavery affects millions of people worldwide, and regulatory frameworks such as Modern Slavery Acts now require companies to publish detailed disclosures.<n>These statements are often vague and inconsistent, making manual review time-consuming and difficult to scale.<n>We propose a novel framework that harnesses AI for rule-level compliance verification while preserving expert oversight.
arXiv Detail & Related papers (2025-11-11T03:41:44Z)
Soppia: A Structured Prompting Framework for the Proportional Assessment of Non-Pecuniary Damages in Personal Injury Cases [0.0]
This paper introduces Soppia, a structured prompting framework designed to assist legal professionals in navigating complex legal rules.<n>Using the twelve criteria for non-pecuniary damages established in the Brazilian CLT (Art. 223-G) as a case study, we demonstrate how Soppia operationalizes nuanced legal commands into a practical, replicable, and transparent methodology.
arXiv Detail & Related papers (2025-10-24T01:42:38Z)
RIRAG: Regulatory Information Retrieval and Answer Generation [51.998738311700095]
We introduce a task of generating question-passages pairs, where questions are automatically created and paired with relevant regulatory passages.<n>We create the ObliQA dataset, containing 27,869 questions derived from the collection of Abu Dhabi Global Markets (ADGM) financial regulation documents.<n>We design a baseline Regulatory Information Retrieval and Answer Generation (RIRAG) system and evaluate it with RePASs, a novel evaluation metric.
arXiv Detail & Related papers (2024-09-09T14:44:19Z)
The Decisive Power of Indecision: Low-Variance Risk-Limiting Audits and Election Contestation via Marginal Mark Recording [51.82772358241505]
Risk-limiting audits (RLAs) are techniques for verifying the outcomes of large elections. We define new families of audits that improve efficiency and offer advances in statistical power. New audits are enabled by revisiting the standard notion of a cast-vote record so that it can declare multiple possible mark interpretations.
arXiv Detail & Related papers (2024-02-09T16:23:54Z)
Towards a multi-stakeholder value-based assessment framework for algorithmic systems [76.79703106646967]
We develop a value-based assessment framework that visualizes closeness and tensions between values. We give guidelines on how to operationalize them, while opening up the evaluation and deliberation process to a wide range of stakeholders.
arXiv Detail & Related papers (2022-05-09T19:28:32Z)
Equality before the Law: Legal Judgment Consistency Analysis for Fairness [55.91612739713396]
In this paper, we propose an evaluation metric for judgment inconsistency, Legal Inconsistency Coefficient (LInCo) We simulate judges from different groups with legal judgment prediction (LJP) models and measure the judicial inconsistency with the disagreement of the judgment results given by LJP models trained on different groups. We employ LInCo to explore the inconsistency in real cases and come to the following observations: (1) Both regional and gender inconsistency exist in the legal system, but gender inconsistency is much less than regional inconsistency.
arXiv Detail & Related papers (2021-03-25T14:28:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.