DiffuSyn Bench: Evaluating Vision-Language Models on Real-World Complexities with Diffusion-Generated Synthetic Benchmarks

• URL: http://arxiv.org/abs/2406.04470v2
• Date: Thu, 13 Jun 2024 16:46:22 GMT
• Title: DiffuSyn Bench: Evaluating Vision-Language Models on Real-World Complexities with Diffusion-Generated Synthetic Benchmarks
• Authors: Haokun Zhou, Yipeng Hong,
• Abstract summary: This study assesses the ability of Large Vision-Language Models (LVLMs) to differentiate between AI-generated and human-generated images. It introduces a new automated benchmark construction method for this evaluation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study assesses the ability of Large Vision-Language Models (LVLMs) to differentiate between AI-generated and human-generated images. It introduces a new automated benchmark construction method for this evaluation. The experiment compared common LVLMs with human participants using a mixed dataset of AI and human-created images. Results showed that LVLMs could distinguish between the image types to some extent but exhibited a rightward bias, and perform significantly worse compared to humans. To build on these findings, we developed an automated benchmark construction process using AI. This process involved topic retrieval, narrative script generation, error embedding, and image generation, creating a diverse set of text-image pairs with intentional errors. We validated our method through constructing two caparable benchmarks. This study highlights the strengths and weaknesses of LVLMs in real-world understanding and advances benchmark construction techniques, providing a scalable and automatic approach for AI model evaluation.

Related papers

• Detecting the Undetectable: Combining Kolmogorov-Arnold Networks and MLP for AI-Generated Image Detection [0.0]
  This paper presents a novel detection framework adept at robustly identifying images produced by cutting-edge generative AI models. We propose a classification system that integrates semantic image embeddings with a traditional Multilayer Perceptron (MLP)
  arXiv  Detail & Related papers  (2024-08-18T06:00:36Z)
• A Sanity Check for AI-generated Image Detection [49.08585395873425]
  We present a sanity check on whether the task of AI-generated image detection has been solved. To quantify the generalization of existing methods, we evaluate 9 off-the-shelf AI-generated image detectors on Chameleon dataset. We propose AIDE (AI-generated Image DEtector with Hybrid Features), which leverages multiple experts to simultaneously extract visual artifacts and noise patterns.
  arXiv  Detail & Related papers  (2024-06-27T17:59:49Z)
• Improving Interpretability and Robustness for the Detection of AI-Generated Images [6.116075037154215]
  We analyze existing state-of-the-art AIGI detection methods based on frozen CLIP embeddings. We show how to interpret them, shedding light on how images produced by various AI generators differ from real ones.
  arXiv  Detail & Related papers  (2024-06-21T10:33:09Z)
• RIGID: A Training-free and Model-Agnostic Framework for Robust AI-Generated Image Detection [60.960988614701414]
  RIGID is a training-free and model-agnostic method for robust AI-generated image detection. RIGID significantly outperforms existing trainingbased and training-free detectors.
  arXiv  Detail & Related papers  (2024-05-30T14:49:54Z)
• Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
  Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness. Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings. This paper introduces an innovative multi-modal prompt-based methodology for IQA.
  arXiv  Detail & Related papers  (2024-04-23T11:45:32Z)
• Bi-LORA: A Vision-Language Approach for Synthetic Image Detection [14.448350657613364]
  Deep image synthesis techniques, such as generative adversarial networks (GANs) and diffusion models (DMs) have ushered in an era of generating highly realistic images. This paper takes inspiration from the potent convergence capabilities between vision and language, coupled with the zero-shot nature of vision-language models (VLMs) We introduce an innovative method called Bi-LORA that leverages VLMs, combined with low-rank adaptation (LORA) tuning techniques, to enhance the precision of synthetic image detection for unseen model-generated images.
  arXiv  Detail & Related papers  (2024-04-02T13:54:22Z)
• Optimizations of Autoencoders for Analysis and Classification of Microscopic In Situ Hybridization Images [68.8204255655161]
  We propose a deep-learning framework to detect and classify areas of microscopic images with similar levels of gene expression. The data we analyze requires an unsupervised learning model for which we employ a type of Artificial Neural Network - Deep Learning Autoencoders.
  arXiv  Detail & Related papers  (2023-04-19T13:45:28Z)
• Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images [60.34381768479834]
  Recent advancements in diffusion models have enabled the generation of realistic deepfakes from textual prompts in natural language. We pioneer a systematic study on deepfake detection generated by state-of-the-art diffusion models.
  arXiv  Detail & Related papers  (2023-04-02T10:25:09Z)
• CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Images [7.868449549351487]
  This article proposes to enhance our ability to recognise AI-generated images through computer vision. The two sets of data present as a binary classification problem with regard to whether the photograph is real or generated by AI. This study proposes the use of a Convolutional Neural Network (CNN) to classify the images into two categories; Real or Fake.
  arXiv  Detail & Related papers  (2023-03-24T16:33:06Z)
• IMAGINE: Image Synthesis by Image-Guided Model Inversion [79.4691654458141]
  We introduce an inversion based method, denoted as IMAge-Guided model INvErsion (IMAGINE), to generate high-quality and diverse images. We leverage the knowledge of image semantics from a pre-trained classifier to achieve plausible generations. IMAGINE enables the synthesis procedure to simultaneously 1) enforce semantic specificity constraints during the synthesis, 2) produce realistic images without generator training, and 3) give users intuitive control over the generation process.
  arXiv  Detail & Related papers  (2021-04-13T02:00:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.