Efficient and Interpretable Transformer for Counterfactual Fairness

• URL: http://arxiv.org/abs/2604.26188v1
• Date: Wed, 29 Apr 2026 00:20:22 GMT
• Title: Efficient and Interpretable Transformer for Counterfactual Fairness
• Abstract summary: This work bridges a critical gap between fairness theory and machine learning models.<n>It offers a practical framework for responsible AI in high-stakes, highly regulated domains.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The growing reliance of machine learning models in high-stakes, highly regulated domains such as finance and insurance has created a growing tension between predictive performance, interpretability, and regulatory fairness requirements. In these settings, models are expected not only to deliver reliable predictions but also to provide transparent decision rationales and comply with strict fairness requirements. Attention-based transformers offer powerful mechanisms for modeling complex data relationships as demonstrated in various language tasks, yet their attention mechanisms alone do not ensure counterfactually fair predictions, even when combined with fairness-aware techniques. To address these limitations, we propose the Feature Correlation Transformer (FCorrTransformer), an attention-light architecture tailored for tabular data. In this design, the attention matrix admits a direct statistical interpretation as pairwise feature dependencies, enhancing both interpretability and efficiency. Leveraging this structure, we introduce Counterfactual Attention Regularization (CAR), a framework that enforces group-invariant fair representations of sensitive features at the attention level, promoting counterfactually fair predictions without relying on explicit causal assumptions. Empirical evaluations on imbalanced classification and regression benchmarks demonstrate that FCorrTransformer combined with CAR achieves strong counterfactual fairness while maintaining competitive predictive performance and substantially reducing model complexity compared with standard transformer-based baselines. Overall, this work bridges a critical gap between fairness theory and machine learning models, offering a practical framework for responsible AI in regulatory-sensitive domains.

Related papers

• Adaptive Financial Transformer with Regime-Gated Attention for Stock Return Prediction [0.0]
  Adaptive Financial Transformer (AFT) is proposed for stock return prediction under non-stationary financial markets.<n>The model incorporates a Market Regime, an Adaptive Gate Network, and an Adaptive Financial Context module.<n>Results demonstrate competitive predictive performance while reducing model complexity by 15.2%.
  arXiv  Detail & Related papers  (2026-06-28T11:33:19Z)
• Large Vision-Language Models Get Lost in Attention [51.851592109135716]
  We propose a unified framework grounded in information theory and geometry to quantify the geometric and entropic nature of residual updates.<n>Applying this unified framework reveals a fundamental functional decoupling: Attention acts as a subspace-language operator focused on reconfiguration, whereas FFNs serve as subspace-language operators driving semantic innovation.
  arXiv  Detail & Related papers  (2026-05-07T04:45:52Z)
• Cauchy-Schwarz Fairness Regularizer [17.898277374771254]
  Group fairness in machine learning is often enforced by adding a regularizer that reduces the dependence between model predictions and sensitive attributes.<n>We propose a Cauchy-Schwarz fairness regularizer that penalizes the empirical CS divergence between prediction distributions conditioned on sensitive groups.
  arXiv  Detail & Related papers  (2025-12-10T09:39:30Z)
• Why Generate When You Can Transform? Unleashing Generative Attention for Dynamic Recommendation [9.365893765448366]
  Sequential Recommendation (SR) focuses on personalizing user experiences by predicting future preferences based on historical interactions.<n> Transformer models, with their attention mechanisms, have become the dominant architecture in SR tasks.<n>We introduce two generative attention models for SR, each grounded in the principles of Variational Autoencoders (VAE) and Diffusion Models (DMs)
  arXiv  Detail & Related papers  (2025-08-04T04:33:26Z)
• Transformers Learn Faster with Semantic Focus [57.97235825738412]
  We study sparse transformers in terms of learnability and generalization.<n>We find that input-dependent sparse attention models appear to converge faster and generalize better than standard attention models.
  arXiv  Detail & Related papers  (2025-06-17T01:19:28Z)
• FAIR-SIGHT: Fairness Assurance in Image Recognition via Simultaneous Conformal Thresholding and Dynamic Output Repair [4.825037489691159]
  We introduce a post-hoc framework designed to ensure fairness in computer vision systems by combining conformal prediction with a dynamic output repair mechanism.<n>Our approach calculates a fairness-aware non-conformity score that simultaneously assesses prediction errors and fairness violations.<n>When the non-conformity score for a new image exceeds the threshold, FAIR-SIGHT implements targeted corrective adjustments, such as logit shifts for classification and confidence recalibration for detection.
  arXiv  Detail & Related papers  (2025-04-10T02:23:06Z)
• Partial Transportability for Domain Generalization [56.37032680901525]
  Building on the theory of partial identification and transportability, this paper introduces new results for bounding the value of a functional of the target distribution.<n>Our contribution is to provide the first general estimation technique for transportability problems.<n>We propose a gradient-based optimization scheme for making scalable inferences in practice.
  arXiv  Detail & Related papers  (2025-03-30T22:06:37Z)
• Latenrgy: Model Agnostic Latency and Energy Consumption Prediction for Binary Classifiers [0.0]
  Machine learning systems increasingly drive innovation across scientific fields and industry.<n>Yet challenges in compute overhead, specifically during inference, limit their scalability and sustainability.<n>This study addresses critical gaps in the literature, chiefly the lack of generalized predictive techniques for latency and energy consumption.
  arXiv  Detail & Related papers  (2024-12-26T14:51:24Z)
• Advancing Counterfactual Inference through Nonlinear Quantile Regression [77.28323341329461]
  We propose a framework for efficient and effective counterfactual inference implemented with neural networks. The proposed approach enhances the capacity to generalize estimated counterfactual outcomes to unseen data. Empirical results conducted on multiple datasets offer compelling support for our theoretical assertions.
  arXiv  Detail & Related papers  (2023-06-09T08:30:51Z)
• Federated Conformal Predictors for Distributed Uncertainty Quantification [83.50609351513886]
  Conformal prediction is emerging as a popular paradigm for providing rigorous uncertainty quantification in machine learning. In this paper, we extend conformal prediction to the federated learning setting. We propose a weaker notion of partial exchangeability, better suited to the FL setting, and use it to develop the Federated Conformal Prediction framework.
  arXiv  Detail & Related papers  (2023-05-27T19:57:27Z)
• Fairness by Explicability and Adversarial SHAP Learning [0.0]
  We propose a new definition of fairness that emphasises the role of an external auditor and model explicability. We develop a framework for mitigating model bias using regularizations constructed from the SHAP values of an adversarial surrogate model. We demonstrate our approaches using gradient and adaptive boosting on: a synthetic dataset, the UCI Adult (Census) dataset and a real-world credit scoring dataset.
  arXiv  Detail & Related papers  (2020-03-11T14:36:34Z)
• Target-Embedding Autoencoders for Supervised Representation Learning [111.07204912245841]
  This paper analyzes a framework for improving generalization in a purely supervised setting, where the target space is high-dimensional. We motivate and formalize the general framework of target-embedding autoencoders (TEA) for supervised prediction, learning intermediate latent representations jointly optimized to be both predictable from features as well as predictive of targets.
  arXiv  Detail & Related papers  (2020-01-23T02:37:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.