Loupe: A Generalizable and Adaptive Framework for Image Forgery Detection

• URL: http://arxiv.org/abs/2506.16819v1
• Date: Fri, 20 Jun 2025 08:18:44 GMT
• Title: Loupe: A Generalizable and Adaptive Framework for Image Forgery Detection
• Authors: Yuchu Jiang, Jiaming Chu, Jian Zhao, Xin Zhang, Xu Yang, Lei Jin, Chi Zhang, Xuelong Li,
• Abstract summary: We propose Loupe, a lightweight yet effective framework for joint deepfake detection and localization.<n>Loupe integrates a patch-aware classifier and a segmentation module with conditional queries, allowing simultaneous global authenticity classification and fine-grained mask prediction.<n>Our results validate the effectiveness of the proposed patch-level fusion and conditional query design in improving both classification accuracy and spatial localization under diverse forgery patterns.
• Score: 46.442787348123126
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The proliferation of generative models has raised serious concerns about visual content forgery. Existing deepfake detection methods primarily target either image-level classification or pixel-wise localization. While some achieve high accuracy, they often suffer from limited generalization across manipulation types or rely on complex architectures. In this paper, we propose Loupe, a lightweight yet effective framework for joint deepfake detection and localization. Loupe integrates a patch-aware classifier and a segmentation module with conditional queries, allowing simultaneous global authenticity classification and fine-grained mask prediction. To enhance robustness against distribution shifts of test set, Loupe introduces a pseudo-label-guided test-time adaptation mechanism by leveraging patch-level predictions to supervise the segmentation head. Extensive experiments on the DDL dataset demonstrate that Loupe achieves state-of-the-art performance, securing the first place in the IJCAI 2025 Deepfake Detection and Localization Challenge with an overall score of 0.846. Our results validate the effectiveness of the proposed patch-level fusion and conditional query design in improving both classification accuracy and spatial localization under diverse forgery patterns. The code is available at https://github.com/Kamichanw/Loupe.

Related papers

• DiffusionFF: Face Forgery Detection via Diffusion-based Artifact Localization [21.139016641596676]
  DiffusionFF is a novel framework that enhances face forgery detection through diffusion-based artifact localization.<n>Our method utilizes a denoising diffusion model to generate high-quality Structural Dissimilarity (DSSIM) maps, which effectively capture subtle traces of manipulation.
  arXiv  Detail & Related papers  (2025-08-03T18:06:04Z)
• Crane: Context-Guided Prompt Learning and Attention Refinement for Zero-Shot Anomaly Detections [50.343419243749054]
  Anomaly Detection (AD) involves identifying deviations from normal data distributions.<n>We propose a novel approach that conditions the prompts of the text encoder based on image context extracted from the vision encoder.<n>Our method achieves state-of-the-art performance, improving performance by 2% to 29% across different metrics on 14 datasets.
  arXiv  Detail & Related papers  (2025-04-15T10:42:25Z)
• Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detection [23.48106270102081]
  This paper tackles the challenge of detecting partially manipulated facial deepfakes.<n>We leverage the Contrastive Language-Image Pre-training (CLIP) model, specifically its ViT-L/14 visual encoder.<n>The proposed approach utilizes parameter-efficient fine-tuning (PEFT) techniques, such as LN-tuning, to adjust a small subset of the model's parameters.
  arXiv  Detail & Related papers  (2025-03-25T14:10:54Z)
• Weakly-supervised deepfake localization in diffusion-generated images [4.548755617115687]
  We propose a weakly-supervised localization problem based on the Xception network as the backbone architecture. We show that the best performing detection method (based on local scores) is less sensitive to the looser supervision than to the mismatch in terms of dataset or generator.
  arXiv  Detail & Related papers  (2023-11-08T10:27:36Z)
• Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
  Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust. Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model. We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
  arXiv  Detail & Related papers  (2023-07-31T10:22:33Z)
• Detect Any Deepfakes: Segment Anything Meets Face Forgery Detection and Localization [30.317619885984005]
  We introduce the well-trained vision segmentation foundation model, i.e., Segment Anything Model (SAM) in face forgery detection and localization. Based on SAM, we propose the Detect Any Deepfakes (DADF) framework with the Multiscale Adapter. The proposed framework seamlessly integrates end-to-end forgery localization and detection optimization.
  arXiv  Detail & Related papers  (2023-06-29T16:25:04Z)
• DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection [82.8662802404075]
  Existing deepfake detection methods fail to generalize well to unseen or degraded samples.<n>High-level semantics are indispensable recipes for generalizable forgery detection.<n>DeepFake-Adapter is first parameter-efficient tuning approach for deepfake detection.
  arXiv  Detail & Related papers  (2023-06-01T16:23:22Z)
• TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization [17.270110456445806]
  TruFor is a forensic framework that can be applied to a large variety of image manipulation methods. We rely on the extraction of both high-level and low-level traces through a transformer-based fusion architecture. Our method is able to reliably detect and localize both cheapfakes and deepfakes manipulations outperforming state-of-the-art works.
  arXiv  Detail & Related papers  (2022-12-21T11:49:43Z)
• Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection [85.53263670166304]
  One-stage detector basically formulates object detection as dense classification and localization. Recent trend for one-stage detectors is to introduce an individual prediction branch to estimate the quality of localization. This paper delves into the representations of the above three fundamental elements: quality estimation, classification and localization.
  arXiv  Detail & Related papers  (2020-06-08T07:24:33Z)
• Scope Head for Accurate Localization in Object Detection [135.9979405835606]
  We propose a novel detector coined as ScopeNet, which models anchors of each location as a mutually dependent relationship. With our concise and effective design, the proposed ScopeNet achieves state-of-the-art results on COCO.
  arXiv  Detail & Related papers  (2020-05-11T04:00:09Z)
• EHSOD: CAM-Guided End-to-end Hybrid-Supervised Object Detection with Cascade Refinement [53.69674636044927]
  We present EHSOD, an end-to-end hybrid-supervised object detection system. It can be trained in one shot on both fully and weakly-annotated data. It achieves comparable results on multiple object detection benchmarks with only 30% fully-annotated data.
  arXiv  Detail & Related papers  (2020-02-18T08:04:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.