DOCFORGE-BENCH: A Comprehensive Benchmark for Document Forgery Detection and Analysis

• URL: http://arxiv.org/abs/2603.01433v1
• Date: Mon, 02 Mar 2026 04:26:57 GMT
• Title: DOCFORGE-BENCH: A Comprehensive Benchmark for Document Forgery Detection and Analysis
• Authors: Zengqi Zhao, Weidi Xia, Peter Wei, Yan Zhang, Yiyi Zhang, Jane Mo, Tiannan Zhang, Yuanqin Dai, Zexi Chen, Simiao Ren,
• Abstract summary: We present DOCFORGE-BENCH, the first unified zero-shot benchmark for document forgery detection.<n>We evaluate 14 methods across eight datasets spanning text tampering, receipt forgery, and identity document manipulation.<n>Our central finding is a pervasive calibration failure invisible under single-threshold protocols.
• Score: 7.0914801556869
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present DOCFORGE-BENCH, the first unified zero-shot benchmark for document forgery detection, evaluating 14 methods across eight datasets spanning text tampering, receipt forgery, and identity document manipulation. Unlike fine-tuning-oriented evaluations such as ForensicHub [Du et al., 2025], DOCFORGE-BENCH applies all methods with their published pretrained weights and no domain adaptation -- a deliberate design choice that reflects the realistic deployment scenario where practitioners lack labeled document training data. Our central finding is a pervasive calibration failure invisible under single-threshold protocols: methods achieve moderate Pixel-AUC (>=0.76) yet near-zero Pixel-F1. This AUC-F1 gap is not a discrimination failure but a score-distribution shift: tampered regions occupy only 0.27-4.17% of pixels in document images -- an order of magnitude less than in natural image benchmarks -- making the standard tau=0.5 threshold catastrophically miscalibrated. Oracle-F1 is 2-10x higher than fixed-threshold Pixel-F1, confirming that calibration, not representation, is the bottleneck. A controlled calibration experiment validates this: adapting a single threshold on N=10 domain images recovers 39-55% of the Oracle-F1 gap, demonstrating that threshold adaptation -- not retraining -- is the key missing step for practical deployment. Overall, no evaluated method works reliably out-of-the-box on diverse document types, underscoring that document forgery detection remains an unsolved problem. We further note that all eight datasets predate the era of generative AI editing; benchmarks covering diffusion- and LLM-based document forgeries represent a critical open gap on the modern attack surface.

Related papers

• AIForge-Doc: A Benchmark for Detecting AI-Forged Tampering in Financial and Form Documents [7.014776899553499]
  We present AIForge-Doc, the first dedicated benchmark targeting exclusively diffusion-model-based inpainting in financial and form documents with pixel-level annotation.<n>We benchmark three representative detectors -- TruFor, DocTamper, and a zero-shot GPT-4o judge -- and find that all existing methods degrade substantially.
  arXiv  Detail & Related papers  (2026-02-24T05:37:35Z)
• CLUE: Non-parametric Verification from Experience via Hidden-State Clustering [64.50919789875233]
  We show that correctness of a solution is encoded as a geometrically separable signature within the trajectory of hidden activations.<n>ClUE consistently outperforms LLM-as-a-judge baselines and matches or exceeds modern confidence-based methods in reranking candidates.
  arXiv  Detail & Related papers  (2025-10-02T02:14:33Z)
• Crane: Context-Guided Prompt Learning and Attention Refinement for Zero-Shot Anomaly Detection [50.343419243749054]
  Anomaly detection is critical in fields such as medical diagnostics and industrial defect detection.<n> CLIP's coarse-grained image-text alignment limits localization and detection performance for fine-grained anomalies.<n>Crane improves the state-of-the-art ZSAD from 2% to 28%, at both image and pixel levels, while remaining competitive in inference speed.
  arXiv  Detail & Related papers  (2025-04-15T10:42:25Z)
• Rethinking Image Forgery Detection via Soft Contrastive Learning and Unsupervised Clustering [27.495469888054032]
  Image forgery detection aims to detect and locate forged regions in an image.<n>Most existing forgery detection algorithms formulate classification problems to classify pixels into forged or pristine.<n>We propose FOrensic ContrAstive cLustering (FOCAL) method, a novel, simple yet very effective paradigm based on soft contrastive learning and unsupervised clustering.
  arXiv  Detail & Related papers  (2023-08-18T05:05:30Z)
• DocDiff: Document Enhancement via Residual Diffusion Models [7.972081359533047]
  We propose DocDiff, a diffusion-based framework specifically designed for document enhancement problems. DocDiff consists of two modules: the Coarse Predictor (CP) and the High-Frequency Residual Refinement (HRR) module. Our proposed HRR module in pre-trained DocDiff is plug-and-play and ready-to-use, with only 4.17M parameters.
  arXiv  Detail & Related papers  (2023-05-06T01:41:10Z)
• Deep Unrestricted Document Image Rectification [110.61517455253308]
  We present DocTr++, a novel unified framework for document image rectification. We upgrade the original architecture by adopting a hierarchical encoder-decoder structure for multi-scale representation extraction and parsing. We contribute a real-world test set and metrics applicable for evaluating the rectification quality.
  arXiv  Detail & Related papers  (2023-04-18T08:00:54Z)
• UDoc-GAN: Unpaired Document Illumination Correction with Background Light Prior [128.19212716007794]
  UDoc-GAN is first framework to address the problem of document illumination correction under the unpaired setting. We first predict the ambient light features of the document. Then, according to the characteristics of different level of ambient lights, we re-formulate the cycle consistency constraint. Compared with the state-of-the-art approaches, our method demonstrates promising performance in terms of character error rate (CER) and edit distance (ED)
  arXiv  Detail & Related papers  (2022-10-15T07:19:23Z)
• Certified Error Control of Candidate Set Pruning for Two-Stage Relevance Ranking [57.42241521034744]
  We propose the concept of certified error control of candidate set pruning for relevance ranking. Our method successfully prunes the first-stage retrieved candidate sets to improve the second-stage reranking speed.
  arXiv  Detail & Related papers  (2022-05-19T16:00:13Z)
• Tune it the Right Way: Unsupervised Validation of Domain Adaptation via Soft Neighborhood Density [125.64297244986552]
  We propose an unsupervised validation criterion that measures the density of soft neighborhoods by computing the entropy of the similarity distribution between points. Our criterion is simpler than competing validation methods, yet more effective.
  arXiv  Detail & Related papers  (2021-08-24T17:41:45Z)
• Approach for Document Detection by Contours and Contrasts [0.0]
  This paper considers arbitrary document detection performed on a mobile device. We propose a modification of the contour-based method, in which the competing contour location hypotheses are ranked according to the contrast between the areas inside and outside the border. The proposed method provides unmatched state-of-the-art performance on the open MIDV-500 dataset, and it demonstrates results comparable with state-of-the-art performance on the SmartDoc dataset.
  arXiv  Detail & Related papers  (2020-08-06T12:44:40Z)
• Can You Read Me Now? Content Aware Rectification using Angle Supervision [14.095728009592763]
  We present CREASE: Content Aware Rectification using Angle Supervision, the first learned method for document rectification. Our method surpasses previous approaches in terms of OCR accuracy, geometric error and visual similarity.
  arXiv  Detail & Related papers  (2020-08-05T16:58:13Z)
• Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection [85.53263670166304]
  One-stage detector basically formulates object detection as dense classification and localization. Recent trend for one-stage detectors is to introduce an individual prediction branch to estimate the quality of localization. This paper delves into the representations of the above three fundamental elements: quality estimation, classification and localization.
  arXiv  Detail & Related papers  (2020-06-08T07:24:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.