Through the Fairness Lens: Experimental Analysis and Evaluation of Entity Matching

• URL: http://arxiv.org/abs/2307.02726v1
• Date: Thu, 6 Jul 2023 02:21:08 GMT
• Title: Through the Fairness Lens: Experimental Analysis and Evaluation of Entity Matching
• Authors: Nima Shahbazi, Nikola Danevski, Fatemeh Nargesian, Abolfazl Asudeh, Divesh Srivastava
• Abstract summary: Algorithmic fairness has become a timely topic to address machine bias and its societal impacts. Despite extensive research on these two topics, little attention has been paid to the fairness of entity matching. We generate two social datasets for the purpose of auditing EM through the lens of fairness.
• Score: 17.857838691801884
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Entity matching (EM) is a challenging problem studied by different communities for over half a century. Algorithmic fairness has also become a timely topic to address machine bias and its societal impacts. Despite extensive research on these two topics, little attention has been paid to the fairness of entity matching. Towards addressing this gap, we perform an extensive experimental evaluation of a variety of EM techniques in this paper. We generated two social datasets from publicly available datasets for the purpose of auditing EM through the lens of fairness. Our findings underscore potential unfairness under two common conditions in real-world societies: (i) when some demographic groups are overrepresented, and (ii) when names are more similar in some groups compared to others. Among our many findings, it is noteworthy to mention that while various fairness definitions are valuable for different settings, due to EM's class imbalance nature, measures such as positive predictive value parity and true positive rate parity are, in general, more capable of revealing EM unfairness.

Related papers

• Threshold-Independent Fair Matching through Score Calibration [1.5530839016602822]
  We introduce a new approach in Entity Matching (EM) using recent metrics for evaluating biases in score based binary classification. This approach enables the application of various bias metrics like equalized odds, equal opportunity, and demographic parity without depending on threshold settings. This paper contributes to the field of fairness in data cleaning, especially within EM, by promoting a method for generating matching scores that reduce biases across different thresholds.
  arXiv  Detail & Related papers  (2024-05-30T13:37:53Z)
• Intersectional Two-sided Fairness in Recommendation [41.96733939002468]
  We propose a novel approach called Inter-sectional Two-sided Fairness Recommendation (ITFR) Our method utilizes a sharpness-aware loss to perceive disadvantaged groups, and then uses collaborative loss balance to develop consistent distinguishing abilities for different intersectional groups. Our proposed approach effectively alleviates the intersectional two-sided unfairness and consistently outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-05T08:56:24Z)
• Fairness-enhancing mixed effects deep learning improves fairness on in- and out-of-distribution clustered (non-iid) data [6.596656267996196]
  We introduce the Fair Mixed Effects Deep Learning (Fair MEDL) framework. Fair MEDL quantifies cluster-invariant fixed effects (FE) and cluster-specific random effects (RE) We incorporate adversarial debiasing to promote fairness across three key metrics: Equalized Odds, Demographic Parity, and Counterfactual Fairness.
  arXiv  Detail & Related papers  (2023-10-04T20:18:45Z)
• Fairness meets Cross-Domain Learning: a new perspective on Models and Metrics [80.07271410743806]
  We study the relationship between cross-domain learning (CD) and model fairness. We introduce a benchmark on face and medical images spanning several demographic groups as well as classification and localization tasks. Our study covers 14 CD approaches alongside three state-of-the-art fairness algorithms and shows how the former can outperform the latter.
  arXiv  Detail & Related papers  (2023-03-25T09:34:05Z)
• Practical Approaches for Fair Learning with Multitype and Multivariate Sensitive Attributes [70.6326967720747]
  It is important to guarantee that machine learning algorithms deployed in the real world do not result in unfairness or unintended social consequences. We introduce FairCOCCO, a fairness measure built on cross-covariance operators on reproducing kernel Hilbert Spaces. We empirically demonstrate consistent improvements against state-of-the-art techniques in balancing predictive power and fairness on real-world datasets.
  arXiv  Detail & Related papers  (2022-11-11T11:28:46Z)
• Equal Improvability: A New Fairness Notion Considering the Long-term Impact [27.72859815965265]
  We propose a new fairness notion called Equal Improvability (EI) EI equalizes the potential acceptance rate of the rejected samples across different groups. We show that proposed EI-regularized algorithms encourage us to find a fair classifier in terms of EI.
  arXiv  Detail & Related papers  (2022-10-13T04:59:28Z)
• D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias [57.87117733071416]
  We propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases. A user can detect the presence of bias against a group by identifying unfair causal relationships in the causal network. For each interaction, say weakening/deleting a biased causal edge, the system uses a novel method to simulate a new (debiased) dataset.
  arXiv  Detail & Related papers  (2022-08-10T03:41:48Z)
• MultiFair: Multi-Group Fairness in Machine Learning [52.24956510371455]
  We study multi-group fairness in machine learning (MultiFair) We propose a generic end-to-end algorithmic framework to solve it. Our proposed framework is generalizable to many different settings.
  arXiv  Detail & Related papers  (2021-05-24T02:30:22Z)
• Equality before the Law: Legal Judgment Consistency Analysis for Fairness [55.91612739713396]
  In this paper, we propose an evaluation metric for judgment inconsistency, Legal Inconsistency Coefficient (LInCo) We simulate judges from different groups with legal judgment prediction (LJP) models and measure the judicial inconsistency with the disagreement of the judgment results given by LJP models trained on different groups. We employ LInCo to explore the inconsistency in real cases and come to the following observations: (1) Both regional and gender inconsistency exist in the legal system, but gender inconsistency is much less than regional inconsistency.
  arXiv  Detail & Related papers  (2021-03-25T14:28:00Z)
• Through the Data Management Lens: Experimental Analysis and Evaluation of Fair Classification [75.49600684537117]
  Data management research is showing an increasing presence and interest in topics related to data and algorithmic fairness. We contribute a broad analysis of 13 fair classification approaches and additional variants, over their correctness, fairness, efficiency, scalability, and stability. Our analysis highlights novel insights on the impact of different metrics and high-level approach characteristics on different aspects of performance.
  arXiv  Detail & Related papers  (2021-01-18T22:55:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.