FairLay-ML: Intuitive Debugging of Fairness in Data-Driven Social-Critical Software

• URL: http://arxiv.org/abs/2407.01423v2
• Date: Tue, 24 Dec 2024 20:00:36 GMT
• Title: FairLay-ML: Intuitive Debugging of Fairness in Data-Driven Social-Critical Software
• Authors: Normen Yu, Luciana Carreon, Gang Tan, Saeid Tizpaz-Niari,
• Abstract summary: We present FairLay-ML, a tool to test and explain the fairness implications of data-driven solutions.<n>FairLay-ML visualizes the logic of datasets, trained models, and decisions for a given data point.<n>It incorporates counterfactual fairness testing that finds bugs beyond the development datasets.
• Score: 13.530748931794065
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data-driven software solutions have significantly been used in critical domains with significant socio-economic, legal, and ethical implications. The rapid adoptions of data-driven solutions, however, pose major threats to the trustworthiness of automated decision-support software. A diminished understanding of the solution by the developer and historical/current biases in the data sets are primary challenges. To aid data-driven software developers and end-users, we present FairLay-ML, a debugging tool to test and explain the fairness implications of data-driven solutions. FairLay-ML visualizes the logic of datasets, trained models, and decisions for a given data point. In addition, it trains various models with varying fairness-accuracy trade-offs. Crucially, FairLay-ML incorporates counterfactual fairness testing that finds bugs beyond the development datasets. We conducted two studies through FairLay-ML that allowed us to measure false positives/negatives in prevalent counterfactual testing and understand the human perception of counterfactual test cases in a class survey. FairLay-ML and its benchmarks are publicly available at https://github.com/Pennswood/FairLay-ML. The live version of the tool is available at https://fairlayml-v2.streamlit.app/. We provide a video demo of the tool at https://youtu.be/wNI9UWkywVU?t=133.

Related papers

• Hey, That's My Data! Label-Only Dataset Inference in Large Language Models [63.35066172530291]
  CatShift is a label-only dataset-inference framework.<n>It capitalizes on catastrophic forgetting: the tendency of an LLM to overwrite previously learned knowledge when exposed to new data.
  arXiv  Detail & Related papers  (2025-06-06T13:02:59Z)
• Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs [60.881609323604685]
  Large Language Models (LLMs) accessed via black-box APIs introduce a trust challenge.<n>Users pay for services based on advertised model capabilities.<n> providers may covertly substitute the specified model with a cheaper, lower-quality alternative to reduce operational costs.<n>This lack of transparency undermines fairness, erodes trust, and complicates reliable benchmarking.
  arXiv  Detail & Related papers  (2025-04-07T03:57:41Z)
• Training on the Benchmark Is Not All You Need [52.01920740114261]
  We propose a simple and effective data leakage detection method based on the contents of multiple-choice options. Our method is able to work under black-box conditions without access to model training data or weights. We evaluate the degree of data leakage of 31 mainstream open-source LLMs on four benchmark datasets.
  arXiv  Detail & Related papers  (2024-09-03T11:09:44Z)
• FairPFN: Transformers Can do Counterfactual Fairness [41.052676173417574]
  Causal and counterfactual fairness provides an intuitive way to define fairness that closely aligns with legal standards. This study builds upon recent work in in context learning (ICL) and prior fitted networks (PFNs) to learn a transformer called FairPFN. This model is pretrained using synthetic fairness data to eliminate the causal effects of protected attributes directly from observational data.
  arXiv  Detail & Related papers  (2024-07-08T08:36:44Z)
• FairJob: A Real-World Dataset for Fairness in Online Systems [2.3622884172290255]
  We introduce a fairness-aware dataset for job recommendations in advertising. It was collected and prepared to comply with privacy standards and business confidentiality. Despite being anonymized and including a proxy for a sensitive attribute, our dataset preserves predictive power.
  arXiv  Detail & Related papers  (2024-07-03T12:30:39Z)
• AIM: Attributing, Interpreting, Mitigating Data Unfairness [40.351282126410545]
  Existing fair machine learning (FairML) research has predominantly focused on mitigating discriminative bias in the model prediction. We investigate a novel research problem: discovering samples that reflect biases/prejudices from the training data. We propose practical algorithms for measuring and countering sample bias.
  arXiv  Detail & Related papers  (2024-06-13T05:21:10Z)
• Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs [49.57641083688934]
  We introduce a novel approach to anomaly detection in financial data using Large Language Models (LLMs) embeddings. Our experiments demonstrate that LLMs contribute valuable information to anomaly detection as our models outperform the baselines.
  arXiv  Detail & Related papers  (2024-06-05T20:19:09Z)
• Solving Data Quality Problems with Desbordante: a Demo [35.75243108496634]
  Desbordante is an open-source data profiler that aims to close this gap. It is built with emphasis on industrial application: it is efficient, scalable, resilient to crashes, and provides explanations. In this demonstration, we show several scenarios that allow end users to solve different data quality problems.
  arXiv  Detail & Related papers  (2023-07-27T15:26:26Z)
• FairLay-ML: Intuitive Remedies for Unfairness in Data-Driven Social-Critical Algorithms [13.649336187121095]
  This thesis explores whether open-sourced machine learning (ML) model explanation tools can allow a layman to visualize, understand, and suggest intuitive remedies to unfairness in ML-based decision-support systems. This thesis presents FairLay-ML, a proof-of-concept GUI integrating some of the most promising tools to provide intuitive explanations for unfair logic in ML models.
  arXiv  Detail & Related papers  (2023-07-11T06:05:06Z)
• Non-Invasive Fairness in Learning through the Lens of Data Drift [88.37640805363317]
  We show how to improve the fairness of Machine Learning models without altering the data or the learning algorithm. We use a simple but key insight: the divergence of trends between different populations, and, consecutively, between a learned model and minority populations, is analogous to data drift. We explore two strategies (model-splitting and reweighing) to resolve this drift, aiming to improve the overall conformance of models to the underlying data.
  arXiv  Detail & Related papers  (2023-03-30T17:30:42Z)
• Preventing Discriminatory Decision-making in Evolving Data Streams [8.952662914331901]
  Bias in machine learning has rightly received significant attention over the last decade. Most fair machine learning (fair-ML) work to address bias in decision-making systems has focused solely on the offline setting. Despite the wide prevalence of online systems in the real world, work on identifying and correcting bias in the online setting is severely lacking.
  arXiv  Detail & Related papers  (2023-02-16T01:20:08Z)
• D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias [57.87117733071416]
  We propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases. A user can detect the presence of bias against a group by identifying unfair causal relationships in the causal network. For each interaction, say weakening/deleting a biased causal edge, the system uses a novel method to simulate a new (debiased) dataset.
  arXiv  Detail & Related papers  (2022-08-10T03:41:48Z)
• Black-box Dataset Ownership Verification via Backdoor Watermarking [67.69308278379957]
  We formulate the protection of released datasets as verifying whether they are adopted for training a (suspicious) third-party model. We propose to embed external patterns via backdoor watermarking for the ownership verification to protect them. Specifically, we exploit poison-only backdoor attacks ($e.g.$, BadNets) for dataset watermarking and design a hypothesis-test-guided method for dataset verification.
  arXiv  Detail & Related papers  (2022-08-04T05:32:20Z)
• Can We Achieve Fairness Using Semi-Supervised Learning? [13.813788753789428]
  We use semi-supervised techniques to create fair classification models. Our framework, Fair-SSL, takes a very small amount of labeled data as input and generates pseudo-labels for the unlabeled data. Fair-SSL achieves similar performance as three state-of-the-art bias mitigation algorithms.
  arXiv  Detail & Related papers  (2021-11-03T06:47:47Z)
• FreaAI: Automated extraction of data slices to test machine learning models [2.475112368179548]
  We show the feasibility of automatically extracting feature models that result in explainable data slices over which the ML solution under-performs. Our novel technique, IBM FreaAI aka FreaAI, extracts such slices from structured ML test data or any other labeled data.
  arXiv  Detail & Related papers  (2021-08-12T09:21:16Z)
• Hidden Biases in Unreliable News Detection Datasets [60.71991809782698]
  We show that selection bias during data collection leads to undesired artifacts in the datasets. We observed a significant drop (>10%) in accuracy for all models tested in a clean split with no train/test source overlap. We suggest future dataset creation include a simple model as a difficulty/bias probe and future model development use a clean non-overlapping site and date split.
  arXiv  Detail & Related papers  (2021-04-20T17:16:41Z)
• Competency Problems: On Finding and Removing Artifacts in Language Data [50.09608320112584]
  We argue that for complex language understanding tasks, all simple feature correlations are spurious. We theoretically analyze the difficulty of creating data for competency problems when human bias is taken into account.
  arXiv  Detail & Related papers  (2021-04-17T21:34:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.