AEGIS: A Holistic Benchmark for Evaluating Forensic Analysis of AI-Generated Academic Images

• URL: http://arxiv.org/abs/2604.28177v1
• Date: Thu, 30 Apr 2026 17:56:58 GMT
• Title: AEGIS: A Holistic Benchmark for Evaluating Forensic Analysis of AI-Generated Academic Images
• Abstract summary: AEGIS is a holistic benchmark for evaluating forensic analysis of AI-generated academic ImageS.<n>By evaluating 25 leading MLLMs, nine expert models, and one unified multimodal understanding and generation model, AEGIS serves as a diagnostic testbed exposing fundamental limitations in academic image forensics.
• Score: 25.79587906632565
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce AEGIS, A holistic benchmark for Evaluating forensic analysis of AI-Generated academic ImageS. Compared to existing benchmarks, AEGIS features three key advances: (1) Domain-Specific Complexity: covering seven academic categories with 39 fine-grained subtypes, exposing intrinsic forensic difficulty, where even GPT-5.1 reaches 48.80% overall performance and expert models achieve only limited localization accuracy (IoU 30.09%); (2) Diverse Forgery Simulations: modeling four prevalent academic forgery strategies across 25 generative models, with 11 yielding average forensic accuracy below 50%, showing that forensics lag behind generative advances; and (3) Multi-Dimensional Forensic Evaluation: jointly assessing detection, reasoning, and localization, revealing complementary strengths between model families, with multimodal large language models (MLLMs) at 84.74% accuracy in textual artifact recognition and expert detectors peaking at 79.54% accuracy in binary authenticity detection. By evaluating 25 leading MLLMs, nine expert models, and one unified multimodal understanding and generation model, AEGIS serves as a diagnostic testbed exposing fundamental limitations in academic image forensics.

Related papers

• DNA: Dual-stage Native Attribution for Generated Image Source Tracing [75.65470221927096]
  Existing proactive methods rely on watermark embedding or model modification.<n>DNA is a coarse-to-fine framework that follows this hierarchy without additional neural-network training.<n>Experiments show that DNA achieves 89.11% end-to-end attribution accuracy.
  arXiv  Detail & Related papers  (2026-07-15T10:27:52Z)
• Methodology for Creating a Clinically Verified Dermoscopic Image Dataset [0.0]
  This study presents a methodology for constructing a clinically verified dataset of dermatoscopic images for medical informatics research.<n>International collections were primarily created under conditions that differ substantially from routine Russian outpatient practice and mobile dermatoscopy.<n>Using this methodology, a dataset of 1,026 unique dermatoscopic images from 443 patients was collected between June 2025 and May 2026.<n>The dataset includes nine nosological categories; all 39 malignant lesions were histologically verified.
  arXiv  Detail & Related papers  (2026-05-24T16:56:47Z)
• Surg-R1: A Hierarchical Reasoning Foundation Model for Scalable and Interpretable Surgical Decision Support with Multi-Center Clinical Validation [51.897472694590356]
  We present Surg-R1, a surgical Vision-Language Model that addresses the gap through hierarchical reasoning trained via a four-stage pipeline.<n>Our approach introduces three key contributions: (1) a three-level reasoning hierarchy decomposing surgical interpretation into perceptual grounding, relational understanding, and contextual reasoning; and (2) the largest surgical chain-of-thought dataset with 320,000 reasoning pairs.
  arXiv  Detail & Related papers  (2026-03-12T20:26:28Z)
• PanCanBench: A Comprehensive Benchmark for Evaluating Large Language Models in Pancreatic Oncology [48.732366302949515]
  Large language models (LLMs) have achieved expert-level performance on standardized examinations, yet multiple-choice accuracy poorly reflects real-world clinical utility and safety.<n>We developed a human-in-the-loop pipeline to create expert rubrics for de-identified patient questions.<n>We evaluated 22 proprietary and open-source LLMs using an LLM-as-a-judge framework, measuring clinical completeness, factual accuracy, and web-search integration.
  arXiv  Detail & Related papers  (2026-03-02T00:50:39Z)
• How well are open sourced AI-generated image detection models out-of-the-box: A comprehensive benchmark study [5.740397289924559]
  No universal winner exists, with detector rankings exhibiting substantial instability.<n>Our findings challenge the one-size-fits-all'' detector paradigm.
  arXiv  Detail & Related papers  (2026-02-08T04:36:13Z)
• ForensicFormer: Hierarchical Multi-Scale Reasoning for Cross-Domain Image Forgery Detection [0.0]
  ForensicFormer is a hierarchical framework that unifies low-level artifact detection, mid-level boundary analysis, and high-level semantic reasoning.<n>Our method maintains 86.8% average accuracy across seven diverse test sets, spanning traditional manipulations, GAN-generated images, and diffusion model outputs.
  arXiv  Detail & Related papers  (2026-01-12T04:29:36Z)
• Benchmarking foundation models for hyperspectral image classification: Application to cereal crop type mapping [0.9407085421584646]
  This study benchmarks three foundation models for cereal crop mapping using hyperspectral imagery.<n>Performance was measured with overall accuracy (OA), average accuracy (AA), and F1-score.
  arXiv  Detail & Related papers  (2025-10-13T16:21:59Z)
• Beyond Diagnosis: Evaluating Multimodal LLMs for Pathology Localization in Chest Radiographs [33.80781505782195]
  We evaluate two general-purpose large language models (LLMs) and a domain-specific model (MedGemma) in their ability to localize pathologies on chest radiographs.<n>GPT-5 exhibited a localization accuracy of 49.7%, followed by GPT-4 (39.1%) and MedGemma (17.7%), all lower than a task-specific CNN baseline (59.9%) and a radiologist benchmark (80.1%)<n>GPT-4 performed well on pathologies with fixed anatomical locations, but struggled with spatially variable findings and exhibited implausible predictions more frequently.
  arXiv  Detail & Related papers  (2025-09-22T16:54:23Z)
• An Agentic System for Rare Disease Diagnosis with Traceable Reasoning [69.46279475491164]
  We introduce DeepRare, the first rare disease diagnosis agentic system powered by a large language model (LLM)<n>DeepRare generates ranked diagnostic hypotheses for rare diseases, each accompanied by a transparent chain of reasoning.<n>The system demonstrates exceptional diagnostic performance among 2,919 diseases, achieving 100% accuracy for 1013 diseases.
  arXiv  Detail & Related papers  (2025-06-25T13:42:26Z)
• PhyX: Does Your Model Have the "Wits" for Physical Reasoning? [49.083544963243206]
  Existing benchmarks fail to capture a crucial aspect of intelligence: physical reasoning.<n>We introduce PhyX: the first large-scale benchmark designed to assess models capacity for physics-grounded reasoning in visual scenarios.
  arXiv  Detail & Related papers  (2025-05-21T18:33:50Z)
• SpectralEarth: Training Hyperspectral Foundation Models at Scale [47.93167977587301]
  We introduce SpectralEarth, a large-scale multitemporal dataset designed to pretrain hyperspectral foundation models.<n>We pretrain a series of foundation models on SpectralEarth, integrating a spectral adapter into classical vision backbones.<n>In tandem, we construct nine downstream datasets for land-cover, crop-type mapping, and tree-species classification.
  arXiv  Detail & Related papers  (2024-08-15T22:55:59Z)
• Recognize Any Regions [55.76437190434433]
  RegionSpot integrates position-aware localization knowledge from a localization foundation model with semantic information from a ViL model.<n>Experiments in open-world object recognition show that our RegionSpot achieves significant performance gain over prior alternatives.
  arXiv  Detail & Related papers  (2023-11-02T16:31:49Z)
• Improving Visual Grounding by Encouraging Consistent Gradient-based Explanations [58.442103936918805]
  We show that Attention Mask Consistency produces superior visual grounding results than previous methods. AMC is effective, easy to implement, and is general as it can be adopted by any vision-language model.
  arXiv  Detail & Related papers  (2022-06-30T17:55:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.