SynCred-Bench: Benchmarking Synthetic Credibility in AI-Generated Visual Misinformation

• URL: http://arxiv.org/abs/2606.03348v1
• Date: Tue, 02 Jun 2026 08:57:38 GMT
• Title: SynCred-Bench: Benchmarking Synthetic Credibility in AI-Generated Visual Misinformation
• Abstract summary: SYNCRED-Bench is a benchmark of 600 AI-generated misinformation images balanced across six credible-form categories and seven fine-grained circulation styles.<n>Under a 5% false-positive-rate constraint, 15 MLLMs achieve only 10.5% true positive rate (TPR), open-source AIGC detectors achieve less than 5%.<n>Human annotators also struggled to identify synthetic credibility, reaching only 63% TPR.
• Score: 82.22252244442744
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent generative models can now produce visual artifacts with realistic embedded text and layouts, creating a new misinformation threat: synthetic credibility. We introduce SYNCRED-Bench, a benchmark of 600 AI-generated misinformation images balanced across six credible-form categories and seven fine-grained circulation styles, together with FP450, a real-image negative set for measuring false positives. Extensive evaluation shows that existing systems remain unreliable: under a 5% false-positive-rate constraint, 15 MLLMs achieve only 10.5% true positive rate (TPR), open-source AIGC detectors achieve less than 5%, and commercial APIs reach 57.6%. Human annotators also struggled to identify synthetic credibility, reaching only 63% TPR. These findings establish synthetic credibility as a severe and underexplored visual misinformation challenge, and provide a benchmark for developing detectors that reason beyond superficial credibility cues.

Related papers

• NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment [54.4265627247104]
  Large language models (LLMs) are increasingly used in peer review at major AI conferences.<n>Existing benchmarks assess novelty as a single holistic score.<n>We present NovGauge, a human-anchored benchmark for fine-grained novelty assessment diagnosis.
  arXiv  Detail & Related papers  (2026-09-10T08:34:56Z)
• Open-Set Vein Biometric Recognition with Deep Metric Learning [0.0]
  We rigorously evaluate the computational boundaries of Deep Metric Learning (DML) under strict open-set constraints.<n>Our approach learns discnative L2-normalised embeddings and employs prototype-based matching with a similarity threshold.<n>On the large-scale MMCBNU 6000 benchmark, our best model (ResNet50-CBAM) achieves an OSCR of 0.9945, AUROC of 0.9974, and EER of 1.57%, maintaining high identification accuracy (99.6% Rank-1) while robustly rejecting unknown subjects.
  arXiv  Detail & Related papers  (2026-04-16T11:03:11Z)
• Multi-Axis Trust Modeling for Interpretable Account Hijacking Detection [1.0152838128195467]
  This paper proposes a Hadith-inspired multi-axis trust modeling framework.<n>We translate five trust axes - long-term integrity (adalah), behavioral precision (dabt), contextual continuity (isnad), cumulative reputation, and anomaly evidence - into a compact set of 26 semantically meaningful behavioral features for user accounts.<n>We evaluate the framework on the CLUE-LDS cloud activity dataset with injected account hijacking scenarios.
  arXiv  Detail & Related papers  (2026-02-20T19:36:30Z)
• Handcrafted Feature Fusion for Reliable Detection of AI-Generated Images [0.2624902795082451]
  The rapid progress of generative models has enabled the creation of highly realistic synthetic images.<n>Detecting such fake content reliably is an urgent challenge.<n>While deep learning approaches dominate current literature, handcrafted features remain attractive for their interpretability, efficiency, and generalizability.
  arXiv  Detail & Related papers  (2026-01-27T06:43:01Z)
• Predictive Coding and Information Bottleneck for Hallucination Detection in Large Language Models [0.8552050317027305]
  Hallucinations in Large Language Models (LLMs) remain a critical barrier to high-stakes deployment.<n>We introduce [Model Name], a hybrid detection framework that combines neuroscience-inspired signal design with supervised machine learning.
  arXiv  Detail & Related papers  (2026-01-22T05:00:21Z)
• From Memorization to Creativity: LLM as a Designer of Novel Neural-Architectures [48.83701310501069]
  Large language models (LLMs) excel in program synthesis, yet their ability to autonomously navigate neural architecture design-balancing reliability, performance, and structural novelty--remains underexplored.<n>We address this by placing a code-oriented LLM within a closed-loop synthesis framework, analyzing its evolution over 22 supervised fine-tuning cycles.
  arXiv  Detail & Related papers  (2026-01-06T13:20:28Z)
• VISION: Robust and Interpretable Code Vulnerability Detection Leveraging Counterfactual Augmentation [6.576811224645293]
  Graph Neural Networks (GNNs) can learn structural and logical code relationships in a data-driven manner.<n>GNNs often learn'spurious' correlations from superficial code similarities.<n>We propose a unified framework for robust and interpretable vulnerability detection, called VISION.
  arXiv  Detail & Related papers  (2025-08-26T11:20:39Z)
• OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks [52.87238755666243]
  We present OmniEAR, a framework for evaluating how language models reason about physical interactions, tool usage, and multi-agent coordination in embodied tasks.<n>We model continuous physical properties and complex spatial relationships across 1,500 scenarios spanning household and industrial domains.<n>Our systematic evaluation reveals severe performance degradation when models must reason from constraints.
  arXiv  Detail & Related papers  (2025-08-07T17:54:15Z)
• Synchronous Faithfulness Monitoring for Trustworthy Retrieval-Augmented Generation [96.78845113346809]
  Retrieval-augmented language models (RALMs) have shown strong performance and wide applicability in knowledge-intensive tasks. This paper proposes SynCheck, a lightweight monitor that leverages fine-grained decoding dynamics to detect unfaithful sentences. We also introduce FOD, a faithfulness-oriented decoding algorithm guided by beam search for long-form retrieval-augmented generation.
  arXiv  Detail & Related papers  (2024-06-19T16:42:57Z)
• How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts [54.07541591018305]
  We present MAD-Bench, a benchmark that contains 1000 test samples divided into 5 categories, such as non-existent objects, count of objects, and spatial relationship. We provide a comprehensive analysis of popular MLLMs, ranging from GPT-4v, Reka, Gemini-Pro, to open-sourced models, such as LLaVA-NeXT and MiniCPM-Llama3. While GPT-4o achieves 82.82% accuracy on MAD-Bench, the accuracy of any other model in our experiments ranges from 9% to 50%.
  arXiv  Detail & Related papers  (2024-02-20T18:31:27Z)
• Conservative Prediction via Data-Driven Confidence Minimization [70.93946578046003]
  In safety-critical applications of machine learning, it is often desirable for a model to be conservative. We propose the Data-Driven Confidence Minimization framework, which minimizes confidence on an uncertainty dataset.
  arXiv  Detail & Related papers  (2023-06-08T07:05:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.