BIAS-ID: A Framework for Analyzing Transformation Biases in AI-Generated Image Detectors

• URL: http://arxiv.org/abs/2605.31153v1
• Date: Fri, 29 May 2026 11:04:11 GMT
• Title: BIAS-ID: A Framework for Analyzing Transformation Biases in AI-Generated Image Detectors
• Abstract summary: We discuss what it means for a detector to be biased, and how this differs from a lack of robustness.<n>We propose BIAS-ID, a transparent framework for analyzing and quantifying the presence of transformation biases in AI-generated image detectors.<n>Our results highlight the importance of bias-aware evaluation for developing reliable AI-generated image detectors.
• Score: 16.177897817197437
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Given the surge of harmful AI-generated imagery online, reliably distinguishing authentic images from generated ones has become an urgent research topic. While many proposed detection methods perform well under controlled settings, they often collapse when tested on real-world data. A potential root cause are subtle biases in the detectors' training data. As a result, detectors may rely on spurious correlations instead of learning true forensic artifacts. While a recent line of work has identified the problem, there is not yet an established protocol to evaluate how biased a detector actually is. In this work, we therefore take a step back: First, we discuss what it means for a detector to be biased, and how this differs from a lack of robustness. Second, we propose BIAS-ID, a transparent framework for analyzing and quantifying the presence of transformation biases in AI-generated image detectors. We validate our framework by performing an evaluation of six detectors across two datasets, revealing that several state-of-the-art detection methods are strongly affected by biases. Our results highlight the importance of bias-aware evaluation for developing reliable AI-generated image detectors.

Related papers

• Where Detectors Fail: Probing Generative Space for Generalizable AI-Generated Image Detection [69.6123714120735]
  Detecting AI-generated images (AIGI) remains challenging because detectors often fail to generalize to unseen generators.<n>We propose PROBE, a framework that improves detector generalization by actively exploring challenging regions of the generative process.
  arXiv  Detail & Related papers  (2026-05-24T07:15:06Z)
• GenShield: Unified Detection and Artifact Correction for AI-Generated Images [65.11434977803509]
  GenShield is a framework that performs AIGI detection and artifact correction in a closed loop from diagnosis to restoration.<n>A high-quality dataset with large-scale artifact-restored'' pairs is constructed alongside a unified evaluation pipeline.
  arXiv  Detail & Related papers  (2026-05-15T16:06:20Z)
• RAID: A Dataset for Testing the Adversarial Robustness of AI-Generated Image Detectors [57.81012948133832]
  We present RAID (Robust evaluation of AI-generated image Detectors), a dataset of 72k diverse and highly transferable adversarial examples.<n>Our methodology generates adversarial images that transfer with a high success rate to unseen detectors.<n>Our findings indicate that current state-of-the-art AI-generated image detectors can be easily deceived by adversarial examples.
  arXiv  Detail & Related papers  (2025-06-04T14:16:00Z)
• Adversarially Robust AI-Generated Image Detection for Free: An Information Theoretic Perspective [22.514709685678813]
  We show that adversarial training (AT) suffers from performance collapse in AIGI detection.<n>Motivated by this difference, we propose Training-free Robust Detection via Information-theoretic Measures (TRIM)<n>TRIM builds on standard detectors and quantifies feature shifts using prediction entropy and KL divergence.
  arXiv  Detail & Related papers  (2025-05-28T17:20:49Z)
• A Bias-Free Training Paradigm for More General AI-generated Image Detection [15.421102443599773]
  A well-designed forensic detector should detect generator specific artifacts rather than reflect data biases.<n>We propose B-Free, a bias-free training paradigm, where fake images are generated from real ones.<n>We show significant improvements in both generalization and robustness over state-of-the-art detectors.
  arXiv  Detail & Related papers  (2024-12-23T15:54:32Z)
• Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models [68.90917438865078]
  Deepfake techniques for facial synthesis and editing pose serious risks for generative models.<n>In this paper, we investigate how detection performance varies across model backbones, types, and datasets.<n>We introduce Contrastive Blur, which enhances performance on facial images, and MINDER, which addresses noise type bias, balancing performance across domains.
  arXiv  Detail & Related papers  (2024-11-28T13:04:45Z)
• Semi-Truths: A Large-Scale Dataset of AI-Augmented Images for Evaluating Robustness of AI-Generated Image detectors [62.63467652611788]
  We introduce SEMI-TRUTHS, featuring 27,600 real images, 223,400 masks, and 1,472,700 AI-augmented images. Each augmented image is accompanied by metadata for standardized and targeted evaluation of detector robustness. Our findings suggest that state-of-the-art detectors exhibit varying sensitivities to the types and degrees of perturbations, data distributions, and augmentation methods used.
  arXiv  Detail & Related papers  (2024-11-12T01:17:27Z)
• Improving Interpretability and Robustness for the Detection of AI-Generated Images [6.116075037154215]
  We analyze existing state-of-the-art AIGI detection methods based on frozen CLIP embeddings. We show how to interpret them, shedding light on how images produced by various AI generators differ from real ones.
  arXiv  Detail & Related papers  (2024-06-21T10:33:09Z)
• FakeInversion: Learning to Detect Images from Unseen Text-to-Image Models by Inverting Stable Diffusion [18.829659846356765]
  We propose a new synthetic image detector that uses features obtained by inverting an open-source pre-trained Stable Diffusion model. We show that these inversion features enable our detector to generalize well to unseen generators of high visual fidelity. We introduce a new challenging evaluation protocol that uses reverse image search to mitigate stylistic and thematic biases in the detector evaluation.
  arXiv  Detail & Related papers  (2024-06-12T19:14:58Z)
• RIGID: A Training-free and Model-Agnostic Framework for Robust AI-Generated Image Detection [60.960988614701414]
  RIGID is a training-free and model-agnostic method for robust AI-generated image detection. RIGID significantly outperforms existing trainingbased and training-free detectors.
  arXiv  Detail & Related papers  (2024-05-30T14:49:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.