When the Forger Is the Judge: GPT-Image-2 Cannot Recognize Its Own Faked Documents

• URL: http://arxiv.org/abs/2604.25213v1
• Date: Tue, 28 Apr 2026 04:33:26 GMT
• Title: When the Forger Is the Judge: GPT-Image-2 Cannot Recognize Its Own Faked Documents
• Abstract summary: OpenAI's GPT-Image-2 has effectively erased the visual boundary between authentic and AI-edited document images.<n>We release AIForge-Doc v2, a paired dataset of 3,066 GPT-Image-2 document forgeries.
• Score: 8.591475917096318
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: OpenAI's GPT-Image-2 has effectively erased the visual boundary between authentic and AI-edited document images: a single number on a receipt can be replaced in under a second for a few cents. We release AIForge-Doc v2, a paired dataset of 3,066 GPT-Image-2 document forgeries with pixel-precise masks in DocTamper-compatible format, and benchmark four lines of defence: human inspectors (N=120, n=365 pair-votes via the public 2AFC site CanUSpotAI.com), TruFor (generic forensic), DocTamper (qcf-568, document-specific), and the same GPT-Image-2 model as a zero-shot self-judge -- asked, to avoid the trivial "image is mostly real" reading, whether any region was generated or edited by an AI image model. Human 2AFC accuracy is 0.501, indistinguishable from chance: even side-by-side, inspectors cannot tell GPT-Image-2 receipt forgeries from authentic counterparts. The three computational judges sit only modestly above (TruFor 0.599, DocTamper 0.585, self-judge 0.532). The self-judge fails consistently, not by chance: across five prompt strategies and four policies for handling ambiguous responses, AUC never rises above 0.59. To rule out the possibility that the two forensic detectors are broken on our source domain rather than blind to AI inpainting, we calibrate each on a same-domain traditional-tampering set built for its training distribution: TruFor reaches AUC 0.962 on cross-camera splicing of our dataset, DocTamper reaches 0.852 on cross-document OCR-token splicing with two-pass JPEG re-encoding. Both retain near-published performance on traditional tampering; switching to GPT-Image-2 inpainting drops AUC by 0.27-0.36 (0.962->0.599 TruFor; 0.852->0.585 DocTamper), isolating a detection gap specific to GPT-Image-2 inpainting. We release the dataset, pipeline, four-judge protocol, and calibration sets.

Related papers

• From Advertised Improvements to Measured Capabilities: Evaluating ChatGPT Images 2.5 on Forgery Tasks [15.787104302260168]
  After image registration, Flare and Sunburst show fewer OCR-detected changes to surrounding receipt text.<n>Flare retains fewer earlier edits on CORD receipts, while photo-edit sequences provide little separation between models.<n>Fine-print improvements remain unresolved below the OCR reliability limit.
  arXiv  Detail & Related papers  (2026-09-12T00:15:01Z)
• Explaining AI-Image Detection: What the Heatmap Actually Shows [65.01025489527173]
  We build a detector and attach an attribution map as its evidence.<n>We measure what that pair delivers on 186,527 images under controls.<n>Whether an attribution ranking exists at all depends on whether the detector reacts to the image.
  arXiv  Detail & Related papers  (2026-07-31T16:07:05Z)
• Format-Controlled Multi-Scale JPEG Compression Response Analysis for Image-Level Forgery Screening [0.0]
  We propose a lightweight, interpretable feature engineering pipeline for image-level forgery screening using only CPU computation and boosted trees.<n>Our method introduces emphmulti-scale Error Level Analysis (ELA) computed at seven JPEG quality levels, combined with novel emphcross-quality ELA ratio features that capture double-compression artifacts characteristic of spliced regions.
  arXiv  Detail & Related papers  (2026-07-07T08:24:01Z)
• How Far Is Document Parsing from Solved? PureDocBench: A Source-TraceableBenchmark across Clean, Degraded, and Real-World Settings [56.70440596502351]
  The past year has seen over 20 open document parsing models, yet benchmarks almost exclusively on OmniDocBench.<n>We present PureDocBench, a benchmark thatrenders document images from HTML/CSS, covering 10 domains, 66 pages, and 1,475 pages.
  arXiv  Detail & Related papers  (2026-05-08T09:30:31Z)
• GPT-Image-2 in the Wild: A Twitter Dataset of Self-Reported AI-Generated Images from the First Week of Deployment [5.547870160341159]
  GPT-image-2 by OpenAI marks a watershed moment in AI-generated imagery.<n>This dataset is the first published dataset of GPT-image-2 generated images sourced from publicly available Twitter/X posts.<n>We characterize the dataset across four analyses.
  arXiv  Detail & Related papers  (2026-04-28T08:35:51Z)
• DOCFORGE-BENCH: A Comprehensive 0-shot Benchmark for Document Forgery Detection and Analysis [6.329569532501952]
  We present DOCFORGE-BENCH, the first unified zero-shot benchmark for document forgery detection.<n>We evaluate 14 methods across eight datasets spanning text tampering, receipt forgery, and identity document manipulation.<n>Our central finding is a pervasive calibration failure invisible under single-threshold protocols.
  arXiv  Detail & Related papers  (2026-03-02T04:26:57Z)
• AIForge-Doc: A Benchmark for Detecting AI-Forged Tampering in Financial and Form Documents [7.014776899553499]
  We present AIForge-Doc, the first dedicated benchmark targeting exclusively diffusion-model-based inpainting in financial and form documents with pixel-level annotation.<n>We benchmark three representative detectors -- TruFor, DocTamper, and a zero-shot GPT-4o judge -- and find that all existing methods degrade substantially.
  arXiv  Detail & Related papers  (2026-02-24T05:37:35Z)
• Dolphin-v2: Universal Document Parsing via Scalable Anchor Prompting [46.102790941920865]
  We present Dolphin-v2, a two-stage document image parsing model that substantially improves upon the original Dolphin.<n>In the first stage, Dolphin-v2 jointly performs document type classification (digital-born versus photographed) alongside layout analysis.<n>In the second stage, we employ a hybrid parsing strategy: photographed documents are parsed holistically as complete pages to handle geometric distortions, while digital-born documents undergo element-wise parallel parsing guided by the detected layout anchors.
  arXiv  Detail & Related papers  (2026-02-05T07:09:57Z)
• Impact of Labeling Inaccuracy and Image Noise on Tooth Segmentation in Panoramic Radiographs using Federated, Centralized and Local Learning [46.232038247686745]
  Federated learning (FL) may mitigate privacy constraints, heterogeneous data quality, and inconsistent labeling in dental diagnostic AI.<n>We compared FL with centralized (CL) and local learning (LL) for tooth segmentation in panoramic radiographs across multiple data corruption scenarios.
  arXiv  Detail & Related papers  (2025-09-08T11:07:47Z)
• A Vessel Bifurcation Landmark Pair Dataset for Abdominal CT Deformable Image Registration (DIR) Validation [1.906989881803579]
  Deformable image registration (DIR) is an enabling technology in many diagnostic and therapeutic tasks.<n>This dataset is a first-of-its-kind for abdominal DIR validation.<n>The number, accuracy, and distribution of landmark pairs will allow robust validation of DIR algorithms with precision beyond what is currently available.
  arXiv  Detail & Related papers  (2025-01-15T21:28:47Z)
• Deep Unrestricted Document Image Rectification [110.61517455253308]
  We present DocTr++, a novel unified framework for document image rectification. We upgrade the original architecture by adopting a hierarchical encoder-decoder structure for multi-scale representation extraction and parsing. We contribute a real-world test set and metrics applicable for evaluating the rectification quality.
  arXiv  Detail & Related papers  (2023-04-18T08:00:54Z)
• DiT: Self-supervised Pre-training for Document Image Transformer [85.78807512344463]
  We propose DiT, a self-supervised pre-trained Document Image Transformer model. We leverage DiT as the backbone network in a variety of vision-based Document AI tasks. Experiment results have illustrated that the self-supervised pre-trained DiT model achieves new state-of-the-art results.
  arXiv  Detail & Related papers  (2022-03-04T15:34:46Z)
• DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction [99.09177377916369]
  We propose Document Image Transformer (DocTr) to address the issue of geometry and illumination distortion of the document images. Our DocTr achieves 20.02% Character Error Rate (CER), a 15% absolute improvement over the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-25T13:27:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.