GenSyn10: A Multi-Generative AI Dataset For Benchmarking Image Classification

• URL: http://arxiv.org/abs/2607.16283v1
• Date: Fri, 10 Jul 2026 09:47:16 GMT
• Title: GenSyn10: A Multi-Generative AI Dataset For Benchmarking Image Classification
• Abstract summary: GenSyn10 is a CIFAR-10-aligned synthetic image dataset of 60,000 images (10 classes, 32$times$32, 50k/10k split) generated using three state-of-the-art models.<n>We evaluate 17 image classification models under a four-stage protocol: real-data baseline, zero-shot transfer, fine-tuning, and retention.<n>Despite a measurable domain gap, CIFAR-10-trained models achieve up to 96.86% zero-shot accuracy on GenSyn10, increasing to 99.88% after fine-tuning.
• Score: 17.505968172455784
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid advancement of generative AI has outpaced our ability to reliably detect its outputs, particularly when detectors encounter generators they have not seen before. We introduce GenSyn10, a CIFAR-10-aligned synthetic image dataset of 60,000 images (10 classes, 32$\times$32, 50k/10k split) generated using three architecturally diverse state-of-the-art models: FLUX.2-dev (Rectified Flow Transformer), HunyuanImage-3.0 (MoE Transformer), and Qwen-Image-2512 (Multimodal Diffusion Transformer), to advance research in AI-generated image detection. A central challenge in this domain is that detectors perform well on known generators but degrade on unseen ones. GenSyn10 addresses this limitation by curating data from multiple contemporary architectures under a standardized generation protocol, enabling controlled and systematic evaluation of out-of-distribution (OOD) generalization to novel generators. Images are generated using a template-based prompt engine and downsampled to ensure consistency. We evaluate 17 image classification models under a four-stage protocol: real-data baseline, zero-shot transfer, fine-tuning, and retention. Despite a measurable domain gap, CIFAR-10-trained models achieve up to 96.86\% zero-shot accuracy on GenSyn10, increasing to 99.88\% after fine-tuning. In binary real-vs-synthetic classification, fine-tuned models achieve 97-99.9\% accuracy on seen generators but drop to 79-96\% on images from an unseen generator, highlighting persistent limitations in OOD generalization. These results establish GenSyn10 as a controlled benchmark for studying synthetic image detection beyond single-generator settings, supporting research on robustness, domain adaptation, and cross-generator generalization.

Related papers

• Mind the Rift: Cross-Scale Coupling Mismatch for AI-Generated Video Detection [10.264738258468254]
  Cross-scale coupling mismatch is a new forensic signal.<n>In natural videos, macro-level temporal dynamics and micro-level residual patterns are intrinsically coupled by the unified imaging physics pipeline.<n>But AI generators, whose training objectives do not explicitly preserve this joint distribution, systematically violate this coupling.<n>Detecting such mismatch is challenging because it requires independently extracting information at both scales.
  arXiv  Detail & Related papers  (2026-09-01T05:23:48Z)
• LoRC: Detecting AI-Generated Images via Low-Rank Collapse in Semantic Residuals [50.157719466558696]
  We show that modern generators exhibit low-rank collapse in the semantic-residual subspace while preserving the dominant semantic direction.<n>We propose textbfLoRC, a framework that decouples semantic dominance to capture the collapsed residual geometry.<n>Our method improves accuracy by an average of 7.0% across multiple benchmarks and achieves 97.0% accuracy on 39 unseen generators.
  arXiv  Detail & Related papers  (2026-08-21T08:58:47Z)
• DNA: Dual-stage Native Attribution for Generated Image Source Tracing [75.65470221927096]
  Existing proactive methods rely on watermark embedding or model modification.<n>DNA is a coarse-to-fine framework that follows this hierarchy without additional neural-network training.<n>Experiments show that DNA achieves 89.11% end-to-end attribution accuracy.
  arXiv  Detail & Related papers  (2026-07-15T10:27:52Z)
• Ghosts Beneath Textures: Texture-Relation Cues for Cross-Paradigm AI-Generated Image Detection [33.4849821487505]
  ConImageGen is a benchmark for cross-paradigm AI-generated image detection.<n>By suppressing semantic interference, we visualize, for the first time, semantics-irrelevant texture patterns across generation paradigms.<n>We propose DTS-Det, a framework that captures and leverages such relations for generalized AI-generated image detection.
  arXiv  Detail & Related papers  (2026-07-04T13:08:01Z)
• How Fragile Are Training-Free AI-Generated Image Detectors? A Controlled Audit of Score Direction, Preprocessing, and Compression [6.3999417080764225]
  We audit two representative training-free scores -- an autoencoder-reconstruction score (AEROBLADE-style) and a noise-perturbation feature-similarity score (RIGID-style)<n>The audit yields three cautionary findings.
  arXiv  Detail & Related papers  (2026-06-18T17:05:51Z)
• SSAFE: Simple and Strong AI-Generated Image Detection via Frozen Vision Encoders [61.07733730455141]
  We investigate how much information about image authenticity is already encoded in modern multimodal vision representations.<n>We develop a representation-aware data curation strategy that selects a compact set of representative generators for training.<n>Experiments show that combining frozen multimodal representations with carefully curated training data provides a simple and effective approach to AI-generated image detection.
  arXiv  Detail & Related papers  (2026-06-07T13:56:17Z)
• Where Detectors Fail: Probing Generative Space for Generalizable AI-Generated Image Detection [69.6123714120735]
  Detecting AI-generated images (AIGI) remains challenging because detectors often fail to generalize to unseen generators.<n>We propose PROBE, a framework that improves detector generalization by actively exploring challenging regions of the generative process.
  arXiv  Detail & Related papers  (2026-05-24T07:15:06Z)
• Exploiting the Final Component of Generator Architectures for AI-Generated Image Detection [8.799682136894914]
  detector-fine-tuned on the DINOv3 backbone-achieves an average accuracy of 98.83% across 22 testing sets from unseen generators.<n>We propose to "contaminate" real images using the generator's final component and train a detector to distinguish them from the original real images.
  arXiv  Detail & Related papers  (2026-01-28T10:35:05Z)
• FUSE: Unifying Spectral and Semantic Cues for Robust AI-Generated Image Detection [0.2446672595462589]
  FUSE is a hybrid system that combines spectral features extracted through Fast Fourier Transform with semantic features obtained from the CLIP's Vision encoder.<n>It attains 91.36% mean accuracy on the GenImage dataset, 88.71% accuracy across all tested generators, and a mean Average Precision of 94.96%.
  arXiv  Detail & Related papers  (2025-12-25T14:38:39Z)
• Breaking Latent Prior Bias in Detectors for Generalizable AIGC Image Detection [11.907536189598577]
  Current AIGC detectors often achieve near-perfect accuracy on images produced by the same generator used for training but struggle to generalize to outputs from unseen generators.<n>We trace this failure in part to latent prior bias: detectors learn shortcuts tied to patterns stemming from the initial noise vector rather than learning robust generative artifacts.<n>We propose On-Manifold Adversarial Training (OMAT), which generates adversarial examples that remain on the generator's output manifold.
  arXiv  Detail & Related papers  (2025-06-01T07:20:45Z)
• CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AI [58.35348718345307]
  Current efforts to distinguish between real and AI-generated images may lack generalization.<n>We propose a novel framework, Co-Spy, that first enhances existing semantic features.<n>We also create Co-Spy-Bench, a comprehensive dataset comprising 5 real image datasets and 22 state-of-the-art generative models.
  arXiv  Detail & Related papers  (2025-03-24T01:59:29Z)
• Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation [52.509092010267665]
  We introduce LlamaGen, a new family of image generation models that apply original next-token prediction'' paradigm of large language models to visual generation domain. It is an affirmative answer to whether vanilla autoregressive models, e.g., Llama, without inductive biases on visual signals can achieve state-of-the-art image generation performance if scaling properly.
  arXiv  Detail & Related papers  (2024-06-10T17:59:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.