Group Fairness Meets the Black Box: Enabling Fair Algorithms on Closed LLMs via Post-Processing

• URL: http://arxiv.org/abs/2508.11258v1
• Date: Fri, 15 Aug 2025 06:50:29 GMT
• Title: Group Fairness Meets the Black Box: Enabling Fair Algorithms on Closed LLMs via Post-Processing
• Authors: Ruicheng Xian, Yuxuan Wan, Han Zhao,
• Abstract summary: We propose a framework for deriving fair classifiers from closed-weight LLMs via prompting.<n>Our framework is data-efficient and outperforms fair classifiers trained on LLM embeddings.
• Score: 14.622788745587815
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Instruction fine-tuned large language models (LLMs) enable a simple zero-shot or few-shot prompting paradigm, also known as in-context learning, for building prediction models. This convenience, combined with continued advances in LLM capability, has the potential to drive their adoption across a broad range of domains, including high-stakes applications where group fairness -- preventing disparate impacts across demographic groups -- is essential. The majority of existing approaches to enforcing group fairness on LLM-based classifiers rely on traditional fair algorithms applied via model fine-tuning or head-tuning on final-layer embeddings, but they are no longer applicable to closed-weight LLMs under the in-context learning setting, which include some of the most capable commercial models today, such as GPT-4, Gemini, and Claude. In this paper, we propose a framework for deriving fair classifiers from closed-weight LLMs via prompting: the LLM is treated as a feature extractor, and features are elicited from its probabilistic predictions (e.g., token log probabilities) using prompts strategically designed for the specified fairness criterion to obtain sufficient statistics for fair classification; a fair algorithm is then applied to these features to train a lightweight fair classifier in a post-hoc manner. Experiments on five datasets, including three tabular ones, demonstrate strong accuracy-fairness tradeoffs for the classifiers derived by our framework from both open-weight and closed-weight LLMs; in particular, our framework is data-efficient and outperforms fair classifiers trained on LLM embeddings (i.e., head-tuning) or from scratch on raw tabular features.

Related papers

• Large Multimodal Models as General In-Context Classifiers [73.11242790834383]
  In this work, we argue that this answer overlooks an important capability of LMMs: in-context learning.<n>We benchmark state-of-the-art LMMs on diverse datasets for closed-world classification and find that, although their zero-shot performance is lower than CLIP's, LMMs with a few in-context examples can match or even surpass contrastive VLMs with cache-based adapters.<n>We extend this analysis to the open-world setting, where the generative nature of LMMs makes them more suitable for the task.
  arXiv  Detail & Related papers  (2026-02-26T17:08:18Z)
• Nonparametric LLM Evaluation from Preference Data [86.96268870461472]
  We propose a nonparametric statistical framework, DMLEval, for comparing and ranking large language models (LLMs) from preference data.<n>Our framework provides practitioners with powerful, state-of-the-art methods for comparing or ranking LLMs.
  arXiv  Detail & Related papers  (2026-01-29T15:00:07Z)
• Fine-Tuning Causal LLMs for Text Classification: Embedding-Based vs. Instruction-Based Approaches [0.0]
  We explore strategies to fine-tune decoder-only Large Language Models (LLMs) for downstream text classification under resource constraints.<n>Two approaches are investigated: (1) attaching a classification head to a pre-trained causal LLM and fine-tuning on the task, and (2) instruction-tuning the LLM in a prompt->response format for classification.
  arXiv  Detail & Related papers  (2025-12-14T13:02:06Z)
• Large Language Models as Universal Predictors? An Empirical Study on Small Tabular Datasets [0.0]
  Large Language Models (LLMs) can perform predictive tasks over structured inputs without explicit fine-tuning on downstream tasks.<n>We investigate the empirical function approximation capability of LLMs on small-scale structured datasets for classification, regression and clustering tasks.<n>Our findings suggest that LLMs can serve as general-purpose predictive engines for structured data, with clear strengths in classification and significant limitations in regression and clustering.
  arXiv  Detail & Related papers  (2025-08-24T15:00:51Z)
• Latent Factor Models Meets Instructions: Goal-conditioned Latent Factor Discovery without Task Supervision [50.45597801390757]
  Instruct-LF is a goal-oriented latent factor discovery system.<n>It integrates instruction-following ability with statistical models to handle noisy datasets.
  arXiv  Detail & Related papers  (2025-02-21T02:03:08Z)
• LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularization [59.75242204923353]
  We introduce LLM-Lasso, a framework that leverages large language models (LLMs) to guide feature selection in Lasso regression.<n>LLMs generate penalty factors for each feature, which are converted into weights for the Lasso penalty using a simple, tunable model.<n>Features identified as more relevant by the LLM receive lower penalties, increasing their likelihood of being retained in the final model.
  arXiv  Detail & Related papers  (2025-02-15T02:55:22Z)
• Improving LLM Group Fairness on Tabular Data via In-Context Learning [23.53624663038328]
  Large language models (LLMs) can fail to generate predictions that satisfy group fairness, that is, produce equitable outcomes across groups.<n>In this work, we investigate four empirical approaches to improve group fairness.<n>We show the effectiveness of these methods in enhancing demographic parity while maintaining high overall performance.
  arXiv  Detail & Related papers  (2024-12-05T22:23:30Z)
• Debiasing Text Safety Classifiers through a Fairness-Aware Ensemble [2.1450827490014865]
  We present a light-weight, post-processing method for mitigating counterfactual fairness in closed-source text safety classifiers. We introduce two threshold-agnostic metrics to assess the counterfactual fairness of a model, and demonstrate how combining these metrics with Fair Data Reweighting (FDW) helps mitigate biases. Our results show that our approach improves counterfactual fairness with minimal impact on model performance.
  arXiv  Detail & Related papers  (2024-09-05T14:35:35Z)
• Bridging LLMs and KGs without Fine-Tuning: Intermediate Probing Meets Subgraph-Aware Entity Descriptions [49.36683223327633]
  Large Language Models (LLMs) encapsulate extensive world knowledge and exhibit powerful context modeling capabilities.<n>We propose a novel framework that synergizes the strengths of LLMs with robust knowledge representation to enable effective and efficient KGC.<n>We achieve a 47% relative improvement over previous methods based on non-fine-tuned LLMs and, to our knowledge, are the first to achieve classification performance comparable to fine-tuned LLMs.
  arXiv  Detail & Related papers  (2024-08-13T10:15:55Z)
• An Actionable Framework for Assessing Bias and Fairness in Large Language Model Use Cases [0.0]
  Large language models (LLMs) can exhibit bias in a variety of ways.<n>We propose a decision framework that allows practitioners to determine which bias and fairness metrics to use for a specific use case.
  arXiv  Detail & Related papers  (2024-07-15T16:04:44Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• Few-Shot Fairness: Unveiling LLM's Potential for Fairness-Aware Classification [7.696798306913988]
  We introduce a framework outlining fairness regulations aligned with various fairness definitions. We explore the configuration for in-context learning and the procedure for selecting in-context demonstrations using RAG. Experiments conducted with different LLMs indicate that GPT-4 delivers superior results in terms of both accuracy and fairness compared to other models.
  arXiv  Detail & Related papers  (2024-02-28T17:29:27Z)
• Selecting Shots for Demographic Fairness in Few-Shot Learning with Large Language Models [14.772568847965408]
  We explore the effect of shots, which directly affect the performance of models, on the fairness of large language models (LLMs) as NLP classification systems. We consider how different shot selection strategies, both existing and new demographically sensitive methods, affect model fairness across three standard fairness datasets.
  arXiv  Detail & Related papers  (2023-11-14T19:02:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.