Automatic Bias Detection in Source Code Review

• URL: http://arxiv.org/abs/2504.18449v1
• Date: Fri, 25 Apr 2025 16:01:52 GMT
• Title: Automatic Bias Detection in Source Code Review
• Authors: Yoseph Berhanu Alebachew, Chris Brown,
• Abstract summary: We propose a controlled experiment to detect potentially biased outcomes in code reviews by observing how reviewers interact with the code.<n>We employ the "spotlight model of attention", a cognitive framework where a reviewer's gaze is tracked to determine their focus areas on the review screen.<n>We plan to analyze the sequence of gaze focus using advanced sequence modeling techniques, including Markov Models, Recurrent Neural Networks (RNNs), and Conditional Random Fields (CRF)
• Score: 2.3480418671346164
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bias is an inherent threat to human decision-making, including in decisions made during software development. Extensive research has demonstrated the presence of biases at various stages of the software development life-cycle. Notably, code reviews are highly susceptible to prejudice-induced biases, and individuals are often unaware of these biases as they occur. Developing methods to automatically detect these biases is crucial for addressing the associated challenges. Recent advancements in visual data analytics have shown promising results in detecting potential biases by analyzing user interaction patterns. In this project, we propose a controlled experiment to extend this approach to detect potentially biased outcomes in code reviews by observing how reviewers interact with the code. We employ the "spotlight model of attention", a cognitive framework where a reviewer's gaze is tracked to determine their focus areas on the review screen. This focus, identified through gaze tracking, serves as an indicator of the reviewer's areas of interest or concern. We plan to analyze the sequence of gaze focus using advanced sequence modeling techniques, including Markov Models, Recurrent Neural Networks (RNNs), and Conditional Random Fields (CRF). These techniques will help us identify patterns that may suggest biased interactions. We anticipate that the ability to automatically detect potentially biased interactions in code reviews will significantly reduce unnecessary push-backs, enhance operational efficiency, and foster greater diversity and inclusion in software development. This approach not only helps in identifying biases but also in creating a more equitable development environment by mitigating these biases effectively

Related papers

• Unmasking Conversational Bias in AI Multiagent Systems [1.0705399532413618]
  biases that may arise in multi-agent systems involving generative models remain under-researched.<n>We present a framework designed to quantify biases within multi-agent systems of conversational Large Language Models.<n>The bias observed in the echo-chamber experiment remains undetected by current state-of-the-art bias detection methods.
  arXiv  Detail & Related papers  (2025-01-24T09:10:02Z)
• Looking at Model Debiasing through the Lens of Anomaly Detection [11.113718994341733]
  Deep neural networks are sensitive to bias in the data.<n>In this work, we show the importance of accurately predicting the bias-conflicting and bias-aligned samples.<n>We propose a new bias identification method based on anomaly detection.
  arXiv  Detail & Related papers  (2024-07-24T17:30:21Z)
• A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection [52.228708947607636]
  This paper introduces a comprehensive visual anomaly detection benchmark, ADer, which is a modular framework for new methods. The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics. We objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection.
  arXiv  Detail & Related papers  (2024-06-05T13:40:07Z)
• Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
  We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy. We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
  arXiv  Detail & Related papers  (2023-02-22T14:50:24Z)
• Self-supervised debiasing using low rank regularization [59.84695042540525]
  Spurious correlations can cause strong biases in deep neural networks, impairing generalization ability. We propose a self-supervised debiasing framework potentially compatible with unlabeled samples. Remarkably, the proposed debiasing framework significantly improves the generalization performance of self-supervised learning baselines.
  arXiv  Detail & Related papers  (2022-10-11T08:26:19Z)
• D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias [57.87117733071416]
  We propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases. A user can detect the presence of bias against a group by identifying unfair causal relationships in the causal network. For each interaction, say weakening/deleting a biased causal edge, the system uses a novel method to simulate a new (debiased) dataset.
  arXiv  Detail & Related papers  (2022-08-10T03:41:48Z)
• Towards Unbiased Visual Emotion Recognition via Causal Intervention [63.74095927462]
  We propose a novel Emotion Recognition Network (IERN) to alleviate the negative effects brought by the dataset bias. A series of designed tests validate the effectiveness of IERN, and experiments on three emotion benchmarks demonstrate that IERN outperforms other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-07-26T10:40:59Z)
• Towards Learning an Unbiased Classifier from Biased Data via Conditional Adversarial Debiasing [17.113618920885187]
  We present a novel adversarial debiasing method, which addresses a feature that is spuriously connected to the labels of training images. We argue by a mathematical proof that our approach is superior to existing techniques for the abovementioned bias. Our experiments show that our approach performs better than state-of-the-art techniques on a well-known benchmark dataset with real-world images of cats and dogs.
  arXiv  Detail & Related papers  (2021-03-10T16:50:42Z)
• Towards causal benchmarking of bias in face analysis algorithms [54.19499274513654]
  We develop an experimental method for measuring algorithmic bias of face analysis algorithms. Our proposed method is based on generating synthetic transects'' of matched sample images. We validate our method by comparing it to a study that employs the traditional observational method for analyzing bias in gender classification algorithms.
  arXiv  Detail & Related papers  (2020-07-13T17:10:34Z)
• Sequential Anomaly Detection using Inverse Reinforcement Learning [23.554584457413483]
  We propose an end-to-end framework for sequential anomaly detection using inverse reinforcement learning (IRL) We use a neural network to represent a reward function. Using a learned reward function, we evaluate whether a new observation from the target agent follows a normal pattern. The empirical study on publicly available real-world data shows that our proposed method is effective in identifying anomalies.
  arXiv  Detail & Related papers  (2020-04-22T05:17:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.