From Advertised Improvements to Measured Capabilities: Evaluating ChatGPT Images 2.5 on Forgery Tasks

• URL: http://arxiv.org/abs/2609.13617v1
• Date: Sat, 12 Sep 2026 00:15:01 GMT
• Title: From Advertised Improvements to Measured Capabilities: Evaluating ChatGPT Images 2.5 on Forgery Tasks
• Abstract summary: After image registration, Flare and Sunburst show fewer OCR-detected changes to surrounding receipt text.<n>Flare retains fewer earlier edits on CORD receipts, while photo-edit sequences provide little separation between models.<n>Fine-print improvements remain unresolved below the OCR reliability limit.
• Score: 15.787104302260168
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We evaluate whether the improvements advertised for ChatGPT Images 2.5 translate into better performance on forgery tasks with predetermined answers. We compare its Flare and Sunburst API models with GPT-Image-2 re-run in the same week, using receipt-field edits, repeated editing, product placement and fine-print rendering. After image registration, Flare and Sunburst show fewer OCR-detected changes to surrounding receipt text (31.7% and 31.2% versus 44.2% for both GPT-Image-2 baselines), mainly on CORD receipts, without a detectable improvement in target-field correctness. Flare retains fewer earlier edits on CORD receipts, while photo-edit sequences provide little separation between models. Product codes are more often legible with Images 2.5, alongside larger product placement; the analyses do not establish a fidelity gain independent of size. Fine-print improvements remain unresolved below the OCR reliability limit. Refusals are rare and localisation is weak in both generations. At a fixed detection threshold, Community Forensics flags 68.6% of controlled Images 2.5 images averaged across cells, versus 35.9% of self-reported images posted online. These results motivate task-specific evaluation of advertised capabilities and defences, with explicit limits on what automatic checks can establish.

Related papers

• ChatGPT Images 2.5 in the Wild: A Launch-Period Dataset and Detector Evaluation [12.307366979757065]
  ChatGPT Images 2.5 can change its underlying generator while retaining its public name, making version attribution from online posts ambiguous.<n>Our frozen collection contains 3,478 images from 2,440 posts across 8 sources.<n>It records three attribution tiers and retains standalone images after image-form filtering and targeted review.
  arXiv  Detail & Related papers  (2026-09-14T06:25:39Z)
• TIGER-FG: Text-Guided Implicit Fine-Grained Grounding for E-commerce Retrieval [10.927254533551563]
  E-commerce image search often takes a cropped image as the query, while each candidate is represented by full item images and structured text.<n>We propose TIGER-FG, a text-guided implicit fine-grained grounding framework for e-commerce retrieval.
  arXiv  Detail & Related papers  (2026-05-18T14:07:20Z)
• EditRefiner: A Human-Aligned Agentic Framework for Image Editing Refinement [76.76247443244293]
  EditRefiner is an agentic framework that reformulates post-editing correction as a human-like perception-reasoning-action-evaluation loop.<n>It consistently outperforms state-of-the-art methods in distortion, diagnose accuracy and human perception alignment.
  arXiv  Detail & Related papers  (2026-05-08T09:05:08Z)
• Benchmarking OCR Pipelines with Adaptive Enhancement for Multi-Domain Retail Bill Digitization [0.0]
  This paper proposes and benchmarks an intelligent, quality-aware adaptive Optical Character Recognition pipeline for retail bill digitization.<n> Experiments were conducted on a real-world dataset of 360 heterogeneous retail bill images.<n>The proposed pipeline achieves a Character Error Rate (CER) of 18.4% and Word Error Rate (WER) of 27.6%, representing improvements of 26.4% and 31.2% respectively over the Raw Tesseract baseline.
  arXiv  Detail & Related papers  (2026-04-28T03:31:27Z)
• GEditBench v2: A Human-Aligned Benchmark for General Image Editing [58.86807672117726]
  We introduce GEditBench v2, a comprehensive benchmark with 1,200 real-world user queries spanning 23 tasks.<n>We also propose PVC-Judge, an open-source pairwise assessment model for visual consistency, trained via two novel region-decoupled preference data synthesis pipelines.<n> PVC-Judge achieves state-of-the-art evaluation performance among open-source models and even surpasses GPT-5.1 on average.
  arXiv  Detail & Related papers  (2026-03-30T15:08:32Z)
• PinPoint: Evaluation of Composed Image Retrieval with Explicit Negatives, Multi-Image Queries, and Paraphrase Testing [3.889218009169166]
  Composed Image Retrieval (CIR) has made significant progress, yet current benchmarks are limited to single ground-truth answers.<n>We present PinPoint, a comprehensive real world benchmark with 7,635 queries and 329K relevance judgments.
  arXiv  Detail & Related papers  (2026-03-04T20:55:30Z)
• RAVID: Retrieval-Augmented Visual Detection: A Knowledge-Driven Approach for AI-Generated Image Identification [14.448350657613368]
  RAVID is the first framework for AI-generated image detection that leverages visual retrieval-augmented generation (RAG)<n>Our approach utilizes a fine-tuned CLIP image encoder, RAVID CLIP, enhanced with category-related prompts to improve representation learning.<n> RAVID achieves an average accuracy of 80.27% under degradation conditions, compared to 63.44% for the state-of-the-art model C2P-CLIP.
  arXiv  Detail & Related papers  (2025-08-05T23:10:56Z)
• TAG-WM: Tamper-Aware Generative Image Watermarking via Diffusion Inversion Sensitivity [76.98973481600002]
  This paper proposes a Tamper-Aware Generative image WaterMarking method named TAG-WM.<n>The proposed method comprises four key modules: a dual-mark joint sampling (DMJS) algorithm for embedding copyright and localization watermarks into the latent space while preserving generative quality.<n>The experimental results demonstrate that TAG-WM achieves state-of-the-art performance in both tampering robustness and localization capability even under distortion.
  arXiv  Detail & Related papers  (2025-06-30T03:14:07Z)
• GIE-Bench: Towards Grounded Evaluation for Text-Guided Image Editing [60.66800567924348]
  We introduce a new benchmark designed to evaluate text-guided image editing models.<n>The benchmark includes over 1000 high-quality editing examples across 20 diverse content categories.<n>We conduct a large-scale study comparing GPT-Image-1 against several state-of-the-art editing models.
  arXiv  Detail & Related papers  (2025-05-16T17:55:54Z)
• Towards Dataset Copyright Evasion Attack against Personalized Text-to-Image Diffusion Models [52.877452505561706]
  We propose the first copyright evasion attack specifically designed to undermine dataset ownership verification (DOV)<n>Our CEAT2I comprises three stages: watermarked sample detection, trigger identification, and efficient watermark mitigation.<n>Our experiments show that our CEAT2I effectively evades DOV mechanisms while preserving model performance.
  arXiv  Detail & Related papers  (2025-05-05T17:51:55Z)
• Doubly Abductive Counterfactual Inference for Text-based Image Editing [130.46583155383735]
  We study text-based image editing (TBIE) of a single image by counterfactual inference. We propose a Doubly Abductive Counterfactual inference framework (DAC) Our DAC achieves a good trade-off between editability and fidelity.
  arXiv  Detail & Related papers  (2024-03-05T13:59:21Z)
• Revisiting Consistency Regularization for Semi-supervised Change Detection in Remote Sensing Images [60.89777029184023]
  We propose a semi-supervised CD model in which we formulate an unsupervised CD loss in addition to the supervised Cross-Entropy (CE) loss. Experiments conducted on two publicly available CD datasets show that the proposed semi-supervised CD method can reach closer to the performance of supervised CD.
  arXiv  Detail & Related papers  (2022-04-18T17:59:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.