TAP into the Patch Tokens: Leveraging Vision Foundation Model Features for AI-Generated Image Detection

• URL: http://arxiv.org/abs/2604.26772v1
• Date: Wed, 29 Apr 2026 15:03:25 GMT
• Title: TAP into the Patch Tokens: Leveraging Vision Foundation Model Features for AI-Generated Image Detection
• Abstract summary: We present a comprehensive benchmark across multiple vision foundation models (VFMs)<n>We evaluate their out-of-the-box performance for detecting fully-generated AI-images and AI-inpainted images.<n>We propose a simple redesign of the classifier head by utilizing tunable attention pooling (TAP), which aggregates output tokens into a refined global representation.
• Score: 0.10705399532413612
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent methods demonstrate that large-scale pretrained models, such as CLIP vision transformers, effectively detect AI-generated images (AIGIs) from unseen generative models when used as feature extractors. Many state-of-the-art methods for AI-generated image detection build upon the original CLIP-ViT to enhance this generalization. Since CLIP's release, numerous vision foundation models (VFMs) have emerged, incorporating architectural improvements and different training paradigms. Despite these advances, their potential for AIGI detection and AI image forensics remains largely unexplored. In this work, we present a comprehensive benchmark across multiple VFM families, covering diverse pretraining objectives, input resolutions, and model scales. We systematically evaluate their out-of-the-box performance for detecting fully-generated AI-images and AI-inpainted images, and discover that the best model outperforms the original CLIP by more than 12% in accuracy, beating established approaches in the process. To fully leverage the features of a modern VFM, we propose a simple redesign of the classifier head by utilizing tunable attention pooling (TAP), which aggregates output tokens into a refined global representation. Integrating TAP with the latest VFMs yields substantial performance gains across several AIGI detection benchmarks, establishing a new state-of-the-art on two challenging benchmarks for in-the-wild detection of AI-generated and -inpainted images.

Related papers

• LHSDet: High-Resolution AI-Generated Image Detection via Visual Question Answering [9.47286721599147]
  We propose a novel high-resolution AI-generated image detector, termed LHSDet.<n>We formulate the AI-generated image detection task as a Visual Question Answering problem.<n>We use a fine-tuned vision-language framework to fully exploit the complementary information between visual and textual modalities.
  arXiv  Detail & Related papers  (2026-08-08T02:13:33Z)
• Can Vision-Language Models Reason about AI Edits in Images? [78.04125956307838]
  Vision-Language Models (VLMs) offer a promising alternative due to their strong visual understanding and reasoning capabilities.<n>We propose a GRPO-based training framework that utilizes simple accuracy and format rewards.<n>Our approach achieves competitive detection and localization performance compared to state-of-the-art image forgery detectors.
  arXiv  Detail & Related papers  (2026-07-30T16:23:40Z)
• Boosting Robust AIGI Detection with LoRA-based Pairwise Training [55.076681464804636]
  Current AIGI detectors perform admirably on clean datasets, but their detection performance decreases when deployed "in the wild"<n>We propose a novel Lo-based Pairwise Training (RA) strategy designed specifically to achieve robust detection for AIGI under severe distortions.
  arXiv  Detail & Related papers  (2026-04-14T05:35:32Z)
• Detecting AI-Generated Images via Distributional Deviations from Real Images [6.615773227400183]
  We propose a Masking-based Pre-trained model Fine-Tuning (MPFT) strategy, which introduces a Texture-Aware Masking (TAM) mechanism to mask textured areas containing generative model-specific patterns during fine-tuning.<n>Our method, fine-tuned with only a minimal number of images, significantly outperforms existing approaches, achieving up to 98.2% and 94.6% average accuracy on the two datasets, respectively.
  arXiv  Detail & Related papers  (2026-01-07T05:00:13Z)
• Self-Supervised AI-Generated Image Detection: A Camera Metadata Perspective [80.10217707456046]
  We introduce a self-supervised approach for detecting AI-generated images that leverages camera metadata.<n>We train a feature extractor solely on camera-captured photographs by classifying categorical EXIF tags.<n>Our detectors deliver strong generalization to in-the-wild samples and robustness to common benign image perturbations.
  arXiv  Detail & Related papers  (2025-12-05T11:53:18Z)
• Bi-Level Optimization for Self-Supervised AI-Generated Face Detection [56.57881725223548]
  We introduce a self-supervised method for AI-generated face detectors based on bi-level optimization.<n>Our detectors significantly outperform existing approaches in both one-class and binary classification settings.
  arXiv  Detail & Related papers  (2025-07-30T16:38:29Z)
• Quality Assessment and Distortion-aware Saliency Prediction for AI-Generated Omnidirectional Images [70.49595920462579]
  This work studies the quality assessment and distortion-aware saliency prediction problems for AIGODIs.<n>We propose two models with shared encoders based on the BLIP-2 model to evaluate the human visual experience and predict distortion-aware saliency for AI-generated omnidirectional images.
  arXiv  Detail & Related papers  (2025-06-27T05:36:04Z)
• MLEP: Multi-granularity Local Entropy Patterns for Universal AI-generated Image Detection [44.40575446607237]
  There is an urgent need for effective methods to detect AI-generated images (AIGI)<n>We propose Multi-granularity Local Entropy Patterns (MLEP), a set of entropy feature maps computed across shuffled small patches over multiple image scaled.<n>MLEP comprehensively captures pixel relationships across dimensions and scales while significantly disrupting image semantics, reducing potential content bias.
  arXiv  Detail & Related papers  (2025-04-18T14:50:23Z)
• Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detection [58.87142367781417]
  A naively trained detector tends to favor overfitting to the limited and monotonous fake patterns, causing the feature space to become highly constrained and low-ranked.<n>One potential remedy is incorporating the pre-trained knowledge within the vision foundation models to expand the feature space.<n>By freezing the principal components and adapting only the remained components, we preserve the pre-trained knowledge while learning fake patterns.
  arXiv  Detail & Related papers  (2024-11-23T19:10:32Z)
• Detecting the Undetectable: Combining Kolmogorov-Arnold Networks and MLP for AI-Generated Image Detection [0.0]
  This paper presents a novel detection framework adept at robustly identifying images produced by cutting-edge generative AI models. We propose a classification system that integrates semantic image embeddings with a traditional Multilayer Perceptron (MLP)
  arXiv  Detail & Related papers  (2024-08-18T06:00:36Z)
• Improving Interpretability and Robustness for the Detection of AI-Generated Images [6.116075037154215]
  We analyze existing state-of-the-art AIGI detection methods based on frozen CLIP embeddings. We show how to interpret them, shedding light on how images produced by various AI generators differ from real ones.
  arXiv  Detail & Related papers  (2024-06-21T10:33:09Z)
• RIGID: A Training-free and Model-Agnostic Framework for Robust AI-Generated Image Detection [60.960988614701414]
  RIGID is a training-free and model-agnostic method for robust AI-generated image detection. RIGID significantly outperforms existing trainingbased and training-free detectors.
  arXiv  Detail & Related papers  (2024-05-30T14:49:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.