IPAD-CLIP: Teaching CLIP to Detect Image Local Perceptual Artifacts

• URL: http://arxiv.org/abs/2605.08664v1
• Date: Sat, 09 May 2026 04:04:41 GMT
• Title: IPAD-CLIP: Teaching CLIP to Detect Image Local Perceptual Artifacts
• Abstract summary: We formalize the Image Perceptual Artifact Detection (IPAD) task to address this gap.<n>Core challenge of IPAD lies in the localized, subtle, and semantically weak nature of these artifacts.<n>We introduce IPAD-CLIP, a novel framework built upon CLIP that enhances artifact discrimination in both textual and visual spaces.
• Score: 43.42573468235177
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Current image quality assessment methods are heavily biased towards global distortions (e.g., noise, blur), neglecting local perceptual artifacts such as ghosting, lens flare, and moire effects. Although significant progress has been made in artifact removal, the fundamental problem of automatic artifact detection remains largely unexplored. In this paper, we formalize the Image Perceptual Artifact Detection (IPAD) task to address this gap. We contribute a benchmark dataset comprising 3,520 artifact images, including 520 real-captured and 3,000 synthetic samples, each paired with pixel-level masks across three representative artifact categories. The core challenge of IPAD lies in the localized, subtle, and semantically weak nature of these artifacts, which makes them prone to missed detection. To overcome this, we introduce IPAD-CLIP, a novel framework built upon CLIP that enhances artifact discrimination in both textual and visual spaces while preserving generalization capabilities. Our key insight is that local artifacts often exhibit strong correlations with specific semantic contexts. Accordingly, we learn artifact-aware text embeddings to explicitly model the object-artifact relationships, resulting in enhanced representations that clear differentiate between clean and artifact prompts. These text embeddings are then used as anchors to shift the visual encoder's attention from high-level semantics to subtle, low-level artifacts. Extensive experiments demonstrate that IPAD-CLIP offers a resource-efficient adaptation of CLIP for detection, significantly outperforming advanced image anomaly detection and manipulation detection methods on our benchmark. To the best of our knowledge, this is the first study addressing multi-class local perceptual artifact detection in terms of both dataset and model.

Related papers

• See and Fix the Flaws: Enabling VLMs and Diffusion Models to Comprehend Visual Artifacts via Agentic Data Synthesis [17.896266572037348]
  ArtiAgent efficiently creates pairs of real and artifact-injected images.<n>It comprises three agents: a perception agent that recognizes entities and subentities from real images, a synthesis agent that introduces artifacts via artifact injection tools, and a curation agent that filters the synthesized artifacts.
  arXiv  Detail & Related papers  (2026-02-24T14:34:13Z)
• Unveiling Perceptual Artifacts: A Fine-Grained Benchmark for Interpretable AI-Generated Image Detection [95.08316274158165]
  X-AIGD provides pixel-level, categorized annotations of perceptual artifacts, spanning low-level distortions, high-level semantics, and cognitive-level counterfactuals.<n>Existing AIGI detectors demonstrate negligible reliance on perceptual artifacts, even at the most basic distortion level.<n>Explicitly aligning model attention with artifact regions can increase the interpretability and generalization of detectors.
  arXiv  Detail & Related papers  (2026-01-27T10:09:17Z)
• Leveraging Hierarchical Image-Text Misalignment for Universal Fake Image Detection [58.927873049646024]
  We show that fake images cannot be properly aligned with corresponding captions compared to real images.<n>We propose a simple yet effective ITEM by leveraging the image-text misalignment in a joint visual-language space as discriminative clues.
  arXiv  Detail & Related papers  (2025-11-01T06:51:14Z)
• Synthesizing Artifact Dataset for Pixel-level Detection [16.31703475992344]
  Artifact detectors enhance the performance of image-generative models by serving as reward models during fine-tuning.<n>We propose an artifact corruption pipeline that automatically injects artifacts into clean, high-quality synthetic images on a predetermined region.<n>The proposed method achieves performance improvements of 13.2% for ConvNeXt and 3.7% for Swin-T, as verified on human-labeled data.
  arXiv  Detail & Related papers  (2025-09-23T21:28:33Z)
• LEGION: Learning to Ground and Explain for Synthetic Image Detection [49.958951540410816]
  We introduce SynthScars, a high-quality and diverse dataset consisting of 12,236 fully synthetic images with human-expert annotations.<n>It features 4 distinct image content types, 3 categories of artifacts, and fine-grained annotations covering pixel-level segmentation, detailed textual explanations, and artifact category labels.<n>We propose LEGION, a multimodal large language model (MLLM)-based image forgery analysis framework that integrates artifact detection, segmentation, and explanation.
  arXiv  Detail & Related papers  (2025-03-19T14:37:21Z)
• Improving Synthetic Image Detection Towards Generalization: An Image Transformation Perspective [45.210030086193775]
  Current synthetic image detection (SID) pipelines are primarily dedicated to crafting universal artifact features.<n>We propose SAFE, a lightweight and effective detector with three simple image transformations.<n>Our pipeline achieves a new state-of-the-art performance, with remarkable improvements of 4.5% in accuracy and 2.9% in average precision against existing methods.
  arXiv  Detail & Related papers  (2024-08-13T09:01:12Z)
• Knowledge-Aware Artifact Image Synthesis with LLM-Enhanced Prompting and Multi-Source Supervision [5.517240672957627]
  We propose a novel knowledge-aware artifact image synthesis approach that brings lost historical objects accurately into their visual forms. Compared to existing approaches, our proposed model produces higher-quality artifact images that align better with the implicit details and historical knowledge contained within written documents.
  arXiv  Detail & Related papers  (2023-12-13T11:03:07Z)
• Perceptual Artifacts Localization for Image Synthesis Tasks [59.638307505334076]
  We introduce a novel dataset comprising 10,168 generated images, each annotated with per-pixel perceptual artifact labels. A segmentation model, trained on our proposed dataset, effectively localizes artifacts across a range of tasks. We propose an innovative zoom-in inpainting pipeline that seamlessly rectifies perceptual artifacts in the generated images.
  arXiv  Detail & Related papers  (2023-10-09T10:22:08Z)
• Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images [60.34381768479834]
  Recent advancements in diffusion models have enabled the generation of realistic deepfakes from textual prompts in natural language. We pioneer a systematic study on deepfake detection generated by state-of-the-art diffusion models.
  arXiv  Detail & Related papers  (2023-04-02T10:25:09Z)
• Identifying Invariant Texture Violation for Robust Deepfake Detection [17.306386179823576]
  We propose the Invariant Texture Learning framework, which only accesses the published dataset with low visual quality. Our method is based on the prior that the microscopic facial texture of the source face is inevitably violated by the texture transferred from the target person.
  arXiv  Detail & Related papers  (2020-12-19T03:02:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.