A Comprehensive Evaluation of the Sensitivity of Density-Ratio Estimation Based Fairness Measurement in Regression

• URL: http://arxiv.org/abs/2508.14576v1
• Date: Wed, 20 Aug 2025 09:54:55 GMT
• Title: A Comprehensive Evaluation of the Sensitivity of Density-Ratio Estimation Based Fairness Measurement in Regression
• Authors: Abdalwahab Almajed, Maryam Tabar, Peyman Najafirad,
• Abstract summary: This paper develops a set of fairness measurement methods with various density-ratio estimation cores.<n>Experiments show that the choice of density-ratio estimation core could significantly affect the outcome of fairness measurement method.<n>These observations suggest major issues with density-ratio estimation based fairness measurement in regression.
• Score: 4.4701415309453285
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The prevalence of algorithmic bias in Machine Learning (ML)-driven approaches has inspired growing research on measuring and mitigating bias in the ML domain. Accordingly, prior research studied how to measure fairness in regression which is a complex problem. In particular, recent research proposed to formulate it as a density-ratio estimation problem and relied on a Logistic Regression-driven probabilistic classifier-based approach to solve it. However, there are several other methods to estimate a density ratio, and to the best of our knowledge, prior work did not study the sensitivity of such fairness measurement methods to the choice of underlying density ratio estimation algorithm. To fill this gap, this paper develops a set of fairness measurement methods with various density-ratio estimation cores and thoroughly investigates how different cores would affect the achieved level of fairness. Our experimental results show that the choice of density-ratio estimation core could significantly affect the outcome of fairness measurement method, and even, generate inconsistent results with respect to the relative fairness of various algorithms. These observations suggest major issues with density-ratio estimation based fairness measurement in regression and a need for further research to enhance their reliability.

Related papers

• Flow-Based Density Ratio Estimation for Intractable Distributions with Applications in Genomics [80.05951561886123]
  We leverage condition-aware flow matching to derive a single dynamical formulation for tracking density ratios along generative trajectories.<n>We demonstrate competitive performance on simulated benchmarks for closed-form ratio estimation, and show that our method supports versatile tasks in single-cell genomics data analysis.
  arXiv  Detail & Related papers  (2026-02-27T17:27:55Z)
• Projection Pursuit Density Ratio Estimation [44.71752951218575]
  Density ratio estimation (DRE) is a paramount task in machine learning, for its broad applications across multiple domains.<n>Parametric methods for estimating the density ratio possibly lead to biased results if models are misspecified.<n>Conventional non-parametric methods suffer from the curse of dimensionality when the dimension of data is large.<n>We propose a novel approach for DRE based on the projection pursuit approximation.
  arXiv  Detail & Related papers  (2025-06-01T07:15:07Z)
• On the Consistency of Fairness Measurement Methods for Regression Tasks [4.4701415309453285]
  This paper studies the consistency of the output of various fairness measurement methods. It finds that while some fairness measurement approaches show strong consistency across various regression tasks, certain methods show a relatively poor consistency in certain regression tasks.
  arXiv  Detail & Related papers  (2024-06-19T16:35:23Z)
• Embedding Trajectory for Out-of-Distribution Detection in Mathematical Reasoning [50.84938730450622]
  We propose a trajectory-based method TV score, which uses trajectory volatility for OOD detection in mathematical reasoning. Our method outperforms all traditional algorithms on GLMs under mathematical reasoning scenarios. Our method can be extended to more applications with high-density features in output spaces, such as multiple-choice questions.
  arXiv  Detail & Related papers  (2024-05-22T22:22:25Z)
• Collaborative Heterogeneous Causal Inference Beyond Meta-analysis [68.4474531911361]
  We propose a collaborative inverse propensity score estimator for causal inference with heterogeneous data. Our method shows significant improvements over the methods based on meta-analysis when heterogeneity increases.
  arXiv  Detail & Related papers  (2024-04-24T09:04:36Z)
• Overcoming Saturation in Density Ratio Estimation by Iterated Regularization [11.244546184962996]
  We show that a class of kernel methods for density ratio estimation suffers from error saturation. We introduce iterated regularization in density ratio estimation to achieve fast error rates.
  arXiv  Detail & Related papers  (2024-02-21T16:02:14Z)
• Sobolev Space Regularised Pre Density Models [51.558848491038916]
  We propose a new approach to non-parametric density estimation that is based on regularizing a Sobolev norm of the density. This method is statistically consistent, and makes the inductive validation model clear and consistent.
  arXiv  Detail & Related papers  (2023-07-25T18:47:53Z)
• Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimation [137.3520153445413]
  A notable gap exists in the evaluation of causal discovery methods, where insufficient emphasis is placed on downstream inference. We evaluate seven established baseline causal discovery methods including a newly proposed method based on GFlowNets. The results of our study demonstrate that some of the algorithms studied are able to effectively capture a wide range of useful and diverse ATE modes.
  arXiv  Detail & Related papers  (2023-07-11T02:58:10Z)
• LEAN-DMKDE: Quantum Latent Density Estimation for Anomaly Detection [0.0]
  The method combines an autoencoder, for learning a low-dimensional representation of the data, with a density-estimation model. The method predicts a degree of normality for new samples based on the estimated density. The experimental results show that the method performs on par with or outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-11-15T21:51:42Z)
• MCD: Marginal Contrastive Discrimination for conditional density estimation [0.0]
  Marginal Contrastive Discrimination, MCD, reformulates the conditional density function into two factors, the marginal density function of the target variable and a ratio of density functions. Our benchmark reveals that our method significantly outperforms in practice existing methods on most density models and regression datasets.
  arXiv  Detail & Related papers  (2022-06-03T14:22:29Z)
• Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions [91.63716984911278]
  We introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks.
  arXiv  Detail & Related papers  (2021-11-11T14:28:12Z)
• A Statistical Analysis of Summarization Evaluation Metrics using Resampling Methods [60.04142561088524]
  We find that the confidence intervals are rather wide, demonstrating high uncertainty in how reliable automatic metrics truly are. Although many metrics fail to show statistical improvements over ROUGE, two recent works, QAEval and BERTScore, do in some evaluation settings.
  arXiv  Detail & Related papers  (2021-03-31T18:28:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.