FraudBench: A Multimodal Benchmark for Detecting AI-Generated Fraudulent Refund Evidence

• URL: http://arxiv.org/abs/2605.08820v1
• Date: Sat, 09 May 2026 09:11:15 GMT
• Title: FraudBench: A Multimodal Benchmark for Detecting AI-Generated Fraudulent Refund Evidence
• Abstract summary: FraudBench is a multimodal benchmark for detecting AI-generated fraudulent refund evidence.<n>It is constructed from real-world user-review evidence across e-commerce, food delivery, and travel-service scenarios.<n>We evaluate MLLMs, specialized AI-generated image detectors, and human participants under the same settings.
• Score: 36.982811727399984
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Artificial Intelligence (AI)-generated images have become increasingly realistic and readily adaptable to concrete real-world claims, creating new challenges for verifying visual evidence. A concrete emerging risk is AI-generated refund fraud, in which manipulated or synthetic images are used to support claims about damaged products, poor delivery conditions, or service-related defects. Existing AI-generated image detection benchmarks mainly evaluate standalone authenticity classification, cross-generator transfer, or forensic localization, leaving claim-conditioned fraudulent evidence detection underexplored. To bridge this gap, we introduce FraudBench, a multimodal benchmark for detecting AI-generated fraudulent refund evidence. FraudBench is constructed from real-world user-review evidence across e-commerce, food delivery, and travel-service scenarios. We curate real evidence images together with their associated review and product metadata, identify genuine damaged and undamaged evidence through MLLM-assisted filtering and human annotation, and synthesize fake-damaged evidence from genuine undamaged reference images using six state-of-the-art image editing and generation models. Using FraudBench, we evaluate MLLMs, specialized AI-generated image detectors, and human participants under the same settings. Experiments show that current MLLMs often recognize real-damaged evidence but fail on many fake-damaged subsets, with fake-damage detection rates (TPR) far below the 50% baseline on most generator subsets. Specialized detectors generally perform better but remain inconsistent across generators and can produce false positives on real-damaged samples, revealing a clear gap between generic AI image detection and reliable claim-conditioned refund-evidence verification.

Related papers

• Training-Free Reconstruction-Based AI-Generated Image Detectors Are Inherently Vulnerable to Adversarial Examples [12.763640037752666]
  We propose two novel attack methods targeted at detectors that leverage autoencoder reconstruction error.<n>We find that by constructing imperceptible adversarial examples, the distance between original and reconstruction can be artificially increased.<n>Our evaluation shows that detection performance is significantly decreased, even if attacked images additionally undergo real-world degradations.
  arXiv  Detail & Related papers  (2026-08-17T14:45:53Z)
• Defake-o3: From Speculative Rationales to Verifiable Evidence for Explainable AIGI Detection [8.599317072232198]
  We present Defake-o3, an AIGI detector that moves from speculative rationales to verifiable evidence.<n>It combines interactive visual search with verifier-guided evidence alignment.<n>Defake-o3 improves both detection accuracy and explanation quality.
  arXiv  Detail & Related papers  (2026-08-17T08:30:32Z)
• Text Over Image: Auditing Multimodal Robustness in Synthetic Medical Image Detection [38.146353416394234]
  We reformulate synthetic medical image detection as an audit of multimodal robustness at the image-record interface.<n>We find that changing the metadata context alone can flip authenticity judgments.<n>We propose an inference-time mitigation pipeline that detects and neutralizes provenance shortcuts without model retraining.
  arXiv  Detail & Related papers  (2026-06-24T04:08:41Z)
• GenShield: Unified Detection and Artifact Correction for AI-Generated Images [65.11434977803509]
  GenShield is a framework that performs AIGI detection and artifact correction in a closed loop from diagnosis to restoration.<n>A high-quality dataset with large-scale artifact-restored'' pairs is constructed alongside a unified evaluation pipeline.
  arXiv  Detail & Related papers  (2026-05-15T16:06:20Z)
• Beyond Artifacts: Real-Centric Envelope Modeling for Reliable AI-Generated Image Detection [29.26836532055224]
  Existing detectors often overfit to generator-specific artifacts and remain sensitive to real-world degradations.<n>We propose Real-centric Envelope Modeling (REM), a new paradigm that shifts detection from learning generator artifacts to modeling the robust distribution of real images.<n>REM achieves an average improvement of 7.5% over state-of-the-art methods, and notably maintains exceptional generalization on the severely degraded RealChain benchmark.
  arXiv  Detail & Related papers  (2025-12-24T04:41:04Z)
• Self-Supervised AI-Generated Image Detection: A Camera Metadata Perspective [80.10217707456046]
  We introduce a self-supervised approach for detecting AI-generated images that leverages camera metadata.<n>We train a feature extractor solely on camera-captured photographs by classifying categorical EXIF tags.<n>Our detectors deliver strong generalization to in-the-wild samples and robustness to common benign image perturbations.
  arXiv  Detail & Related papers  (2025-12-05T11:53:18Z)
• From Evidence to Verdict: An Agent-Based Forensic Framework for AI-Generated Image Detection [19.240335260177382]
  We introduce AIFo (Agent-based Image Forensics), a training-free framework that emulates human forensic investigation through multi-agent collaboration.<n>Unlike conventional methods, our framework employs a set of forensic tools, including reverse image search, metadata extraction, pre-trained classifiers, and VLM analysis.<n>Our comprehensive evaluation spans 6,000 images and challenges real-world scenarios, including images from modern generative platforms and diverse online sources.
  arXiv  Detail & Related papers  (2025-10-31T18:36:49Z)
• Revisiting Reconstruction-based AI-generated Image Detection: A Geometric Perspective [50.83711509908479]
  We introduce the Jacobian-Spectral Lower Bound for reconstruction error from a geometric perspective.<n>We show that real images off the reconstruction manifold exhibit a non-trivial error lower bound, while generated images on the manifold have near-zero error.<n>We propose ReGap, a training-free method that computes dynamic reconstruction error by leveraging structured editing operations.
  arXiv  Detail & Related papers  (2025-10-29T03:45:03Z)
• Semantic-Aware Reconstruction Error for Detecting AI-Generated Images [22.83053631078616]
  We propose a novel representation, namely Semantic-Aware Reconstruction Error (SARE), that measures the semantic difference between an image and its caption-guided reconstruction.<n>SARE provides a robust and discriminative feature for detecting fake images across diverse generative models.<n>We also introduce a fusion module that integrates SARE into the backbone detector via a cross-attention mechanism.
  arXiv  Detail & Related papers  (2025-08-13T04:37:36Z)
• RAID: A Dataset for Testing the Adversarial Robustness of AI-Generated Image Detectors [57.81012948133832]
  We present RAID (Robust evaluation of AI-generated image Detectors), a dataset of 72k diverse and highly transferable adversarial examples.<n>Our methodology generates adversarial images that transfer with a high success rate to unseen detectors.<n>Our findings indicate that current state-of-the-art AI-generated image detectors can be easily deceived by adversarial examples.
  arXiv  Detail & Related papers  (2025-06-04T14:16:00Z)
• Explainable Synthetic Image Detection through Diffusion Timestep Ensembling [30.298198387824275]
  We propose a novel synthetic image detection method that directly utilizes features of intermediately noised images by training an ensemble on multiple noised timesteps.<n>To enhance human comprehension, we introduce a metric-grounded explanation generation and refinement module.<n>Our method achieves state-of-the-art performance with 98.91% and 95.89% detection accuracy on regular and challenging samples respectively.
  arXiv  Detail & Related papers  (2025-03-08T13:04:20Z)
• FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models [62.66610648697744]
  We introduce a taxonomy of generative visual forgery concerning human perception, based on which we collect forgery descriptions in human natural language. FakeBench examines LMMs with four evaluation criteria: detection, reasoning, interpretation and fine-grained forgery analysis. This research presents a paradigm shift towards transparency for the fake image detection area.
  arXiv  Detail & Related papers  (2024-04-20T07:28:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.