Unraveling Machine Behavior by Multi-Level Bias Analysis and Detection: Methodology and Application to Computer Vision

• URL: http://arxiv.org/abs/2607.07236v1
• Date: Wed, 08 Jul 2026 10:18:51 GMT
• Title: Unraveling Machine Behavior by Multi-Level Bias Analysis and Detection: Methodology and Application to Computer Vision
• Abstract summary: This study investigates the presence and propagation of bias within Neural Networks through a comprehensive multi-level analysis.<n>We propose three bias detection approaches: SpaceBias, ActivationBias, and WeightBias.<n>Our proposed techniques provide insight into how biases manifest within the network architecture itself at different levels.
• Score: 10.595890561268735
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This study investigates the presence and propagation of bias within Neural Networks through a comprehensive multi-level analysis spanning the learned latent space, layer activations, and the network's parameters. Based on this taxonomy, we propose three bias detection approaches: 1) SpaceBias (new method), which characterizes the latent space prior to the final classification layer using neighbor-probability distributions and quantifies bias with the two-sample Kolmogorov-Smirnov test on the per-group distributions. 2) ActivationBias (extension of the existing method InsideBias), which analyzes the activations of neural network filters and quantifies bias via a Mann-Whitney U test, based on the observed fact that underrepresented groups exhibit lower activation levels in the final convolutional layers. 3) WeightBias (extension of the existing method IFBiD), which uses a secondary neural network trained to identify biased patterns directly in the parameters of task-specific models. Unlike conventional methods, which assess neural network outcomes and treat the model as a black box, our proposed techniques provide insight into how biases manifest within the network architecture itself at different levels, offering a more nuanced and detailed understanding. Experiments are conducted on two complementary applications: gender classification in the DiveFace dataset (72,000 face images) and digit classification on a colored-MNIST benchmark with controlled bias severity. In total, more than 127,000 models with varying degrees and types of bias were trained and evaluated. The severity sweep shows that the internal disparity, and with it the detection performance, decreases smoothly as the training distribution approaches balance. The results highlight the importance of methods that provide deeper insight into the behavior of AI models.

Related papers

• Implicit Hypothesis Testing and Divergence Preservation in Neural Network Representations [3.641372680589358]
  We study the supervised training dynamics of neural classifiers through the lens of binary hypothesis testing.<n>We model classification as a set of binary tests between class-conditional distributions of representations and empirically show that, along training trajectories, well-generalizing networks increasingly align with Neyman-Pearson optimal decision rules.
  arXiv  Detail & Related papers  (2026-01-28T10:46:44Z)
• Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks [13.983863226803336]
  We show that neural networks trained by gradient descent tend to rely on an average of features for classification.<n>We prove that gradient descent biases the network towards feature averaging, where the weights of each hidden neuron represent an average of the cluster centers.<n>We prove that a two-layer ReLU network can achieve optimal robustness when trained to classify individual features.
  arXiv  Detail & Related papers  (2024-10-14T09:28:32Z)
• Unleashing the power of Neural Collapse for Transferability Estimation [42.09673383041276]
  Well-trained models exhibit the phenomenon of Neural Collapse. We propose a novel method termed Fair Collapse (FaCe) for transferability estimation. FaCe yields state-of-the-art performance on different tasks including image classification, semantic segmentation, and text classification.
  arXiv  Detail & Related papers  (2023-10-09T14:30:10Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• Anomaly Detection in Cybersecurity: Unsupervised, Graph-Based and Supervised Learning Methods in Adversarial Environments [63.942632088208505]
  Inherent to today's operating environment is the practice of adversarial machine learning. In this work, we examine the feasibility of unsupervised learning and graph-based methods for anomaly detection. We incorporate a realistic adversarial training mechanism when training our supervised models to enable strong classification performance in adversarial environments.
  arXiv  Detail & Related papers  (2021-05-14T10:05:10Z)
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning [50.24174211654775]
  We present a novel contrastive self-supervised learning framework for anomaly detection on attributed networks. Our framework fully exploits the local information from network data by sampling a novel type of contrastive instance pair. A graph neural network-based contrastive learning model is proposed to learn informative embedding from high-dimensional attributes and local structure.
  arXiv  Detail & Related papers  (2021-02-27T03:17:20Z)
• LOGAN: Local Group Bias Detection by Clustering [86.38331353310114]
  We argue that evaluating bias at the corpus level is not enough for understanding how biases are embedded in a model. We propose LOGAN, a new bias detection technique based on clustering. Experiments on toxicity classification and object classification tasks show that LOGAN identifies bias in a local region.
  arXiv  Detail & Related papers  (2020-10-06T16:42:51Z)
• Ramifications of Approximate Posterior Inference for Bayesian Deep Learning in Adversarial and Out-of-Distribution Settings [7.476901945542385]
  We show that Bayesian deep learning models on certain occasions marginally outperform conventional neural networks. Preliminary investigations indicate the potential inherent role of bias due to choices of initialisation, architecture or activation functions.
  arXiv  Detail & Related papers  (2020-09-03T16:58:15Z)
• Learning from Failure: Training Debiased Classifier from Biased Classifier [76.52804102765931]
  We show that neural networks learn to rely on spurious correlation only when it is "easier" to learn than the desired knowledge. We propose a failure-based debiasing scheme by training a pair of neural networks simultaneously. Our method significantly improves the training of the network against various types of biases in both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2020-07-06T07:20:29Z)
• InsideBias: Measuring Bias in Deep Networks and Application to Face Gender Biometrics [73.85525896663371]
  This work explores the biases in learning processes based on deep neural network architectures. We employ two gender detection models based on popular deep neural networks. We propose InsideBias, a novel method to detect biased models.
  arXiv  Detail & Related papers  (2020-04-14T15:20:50Z)
• Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss [0.0]
  Neural networks trained to minimize the logistic (a.k.a. cross-entropy) loss with gradient-based methods are observed to perform well in many supervised classification tasks. We analyze the training and generalization behavior of infinitely wide two-layer neural networks with homogeneous activations.
  arXiv  Detail & Related papers  (2020-02-11T15:42:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.