Findings of the Counter Turing Test: AI-Generated Image Detection

• URL: http://arxiv.org/abs/2605.20787v3
• Date: Mon, 25 May 2026 11:44:54 GMT
• Title: Findings of the Counter Turing Test: AI-Generated Image Detection
• Abstract summary: Increasing realism of AI-generated images makes their detection a pressing concern for researchers, policymakers, and industry stakeholders.<n>We present the findings of the Defactify 4.0 workshop, which introduced the Counter Turing Test (CT2) for AI-Generated Image Detection.<n>The competition consisted of two key tasks: (1) binary classification of images as either AI-generated or real and (2) identification of the specific generative model responsible for an AI-generated image.
• Score: 23.358538027776877
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid advancements in generative AI technologies, such as Stable Diffusion, DALL-E, and Midjourney, have significantly transformed the creation of synthetic visual content. While these models enable innovation across industries, they also pose serious challenges, including misinformation, disinformation, and biased content generation. The increasing realism of AI-generated images makes their detection a pressing concern for researchers, policymakers, and industry stakeholders. In this paper, we present the findings of the Defactify 4.0 workshop, which introduced the Counter Turing Test (CT2) for AI-Generated Image Detection. The competition consisted of two key tasks: (1) binary classification of images as either AI-generated or real and (2) identification of the specific generative model responsible for an AI-generated image. To support both tasks, we employed the MS COCOAI dataset, a benchmark of 96000 real and synthetic images generated by five state-of-the-art models alongside real images from MS COCO. Participants employed diverse detection strategies, including convolutional neural networks (CNNs), Vision Transformers (ViTs), frequency-based analysis, contrastive learning, and multimodal techniques. The results demonstrated that while AI-generated images can be detected with high accuracy (F1-score > 0.83), identifying the exact model used remains significantly more challenging (highest F1-score: 0.4986). These findings highlight the need for improved model fingerprinting, adversarial robustness, and real-time detection mechanisms.

Related papers

• GenShield: Unified Detection and Artifact Correction for AI-Generated Images [65.11434977803509]
  GenShield is a framework that performs AIGI detection and artifact correction in a closed loop from diagnosis to restoration.<n>A high-quality dataset with large-scale artifact-restored'' pairs is constructed alongside a unified evaluation pipeline.
  arXiv  Detail & Related papers  (2026-05-15T16:06:20Z)
• Self-Supervised AI-Generated Image Detection: A Camera Metadata Perspective [80.10217707456046]
  We introduce a self-supervised approach for detecting AI-generated images that leverages camera metadata.<n>We train a feature extractor solely on camera-captured photographs by classifying categorical EXIF tags.<n>Our detectors deliver strong generalization to in-the-wild samples and robustness to common benign image perturbations.
  arXiv  Detail & Related papers  (2025-12-05T11:53:18Z)
• Provenance of AI-Generated Images: A Vector Similarity and Blockchain-based Approach [3.632189127068905]
  We propose an embedding-based AI image detection framework to distinguish AI-generated images from real (human-created) ones.<n>Our methodology is built on the hypothesis that AI-generated images demonstrate closer embedding proximity to other AI-generated content.<n>Our results confirm that moderate to high perturbations minimally impact the embedding signatures, with perturbed images maintaining close similarity matches to their original versions.
  arXiv  Detail & Related papers  (2025-10-15T00:49:56Z)
• Bridging the Gap Between Ideal and Real-world Evaluation: Benchmarking AI-Generated Image Detection in Challenging Scenarios [54.07895223545793]
  This paper introduces the Real-World Robustness dataset (RRDataset) for comprehensive evaluation of detection models across three dimensions.<n>RRDataset includes high-quality images from seven major scenarios.<n>We benchmarked 17 detectors and 10 vision-language models (VLMs) on RRDataset and conducted a large-scale human study.
  arXiv  Detail & Related papers  (2025-09-11T06:15:52Z)
• Quality Assessment and Distortion-aware Saliency Prediction for AI-Generated Omnidirectional Images [70.49595920462579]
  This work studies the quality assessment and distortion-aware saliency prediction problems for AIGODIs.<n>We propose two models with shared encoders based on the BLIP-2 model to evaluate the human visual experience and predict distortion-aware saliency for AI-generated omnidirectional images.
  arXiv  Detail & Related papers  (2025-06-27T05:36:04Z)
• Is Artificial Intelligence Generated Image Detection a Solved Problem? [16.9828631287303]
  AIGIBench is a benchmark designed to rigorously evaluate the robustness and generalization capabilities of state-of-the-art AIGI detectors.<n>It simulates real-world challenges through four core tasks: multi-source generalization, robustness to image degradation, sensitivity to data augmentation, and impact of test-time pre-processing.<n>It includes 23 diverse fake image subsets that span both advanced and widely adopted image generation techniques, along with real-world samples collected from social media and AI art platforms.
  arXiv  Detail & Related papers  (2025-05-18T10:00:39Z)
• CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AI [58.35348718345307]
  Current efforts to distinguish between real and AI-generated images may lack generalization.<n>We propose a novel framework, Co-Spy, that first enhances existing semantic features.<n>We also create Co-Spy-Bench, a comprehensive dataset comprising 5 real image datasets and 22 state-of-the-art generative models.
  arXiv  Detail & Related papers  (2025-03-24T01:59:29Z)
• D-Judge: How Far Are We? Assessing the Discrepancies Between AI-synthesized and Natural Images through Multimodal Guidance [19.760989919485894]
  We construct a large-scale multimodal dataset, D-ANI, comprising 5,000 natural images and over 440,000 AIGI samples.<n>We then introduce an AI-Natural Image Discrepancy assessment benchmark (D-Judge) to address the critical question: how far are AI-generated images (AIGIs) from truly realistic images?
  arXiv  Detail & Related papers  (2024-12-23T15:08:08Z)
• A Sanity Check for AI-generated Image Detection [49.08585395873425]
  We propose AIDE (AI-generated Image DEtector with Hybrid Features) to detect AI-generated images.<n>AIDE achieves +3.5% and +4.6% improvements to state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-06-27T17:59:49Z)
• RIGID: A Training-free and Model-Agnostic Framework for Robust AI-Generated Image Detection [60.960988614701414]
  RIGID is a training-free and model-agnostic method for robust AI-generated image detection. RIGID significantly outperforms existing trainingbased and training-free detectors.
  arXiv  Detail & Related papers  (2024-05-30T14:49:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.