Security Benefits and Side Effects of Labeling AI-Generated Images

• URL: http://arxiv.org/abs/2505.22845v1
• Date: Wed, 28 May 2025 20:24:45 GMT
• Title: Security Benefits and Side Effects of Labeling AI-Generated Images
• Authors: Sandra Höltervennhoff, Jonas Ricker, Maike M. Raphael, Charlotte Schwedes, Rebecca Weil, Asja Fischer, Thorsten Holz, Lea Schönherr, Sascha Fahl,
• Abstract summary: We study the implications of labels, including the possibility of mislabeling.<n>We conduct a pre-registered online survey with over 1300 U.S. and EU participants.<n>We find the undesired side effect that human-made images conveying inaccurate claims were perceived as more credible in the presence of labels.
• Score: 27.771584371064968
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative artificial intelligence is developing rapidly, impacting humans' interaction with information and digital media. It is increasingly used to create deceptively realistic misinformation, so lawmakers have imposed regulations requiring the disclosure of AI-generated content. However, only little is known about whether these labels reduce the risks of AI-generated misinformation. Our work addresses this research gap. Focusing on AI-generated images, we study the implications of labels, including the possibility of mislabeling. Assuming that simplicity, transparency, and trust are likely to impact the successful adoption of such labels, we first qualitatively explore users' opinions and expectations of AI labeling using five focus groups. Second, we conduct a pre-registered online survey with over 1300 U.S. and EU participants to quantitatively assess the effect of AI labels on users' ability to recognize misinformation containing either human-made or AI-generated images. Our focus groups illustrate that, while participants have concerns about the practical implementation of labeling, they consider it helpful in identifying AI-generated images and avoiding deception. However, considering security benefits, our survey revealed an ambiguous picture, suggesting that users might over-rely on labels. While inaccurate claims supported by labeled AI-generated images were rated less credible than those with unlabeled AI-images, the belief in accurate claims also decreased when accompanied by a labeled AI-generated image. Moreover, we find the undesired side effect that human-made images conveying inaccurate claims were perceived as more credible in the presence of labels.

Related papers

• AI labeling reduces the perceived accuracy of online content but has limited broader effects [0.0]
  We show that explicit AI labeling of a news article about a proposed public policy reduces its perceived accuracy.<n>We find that AI labeling reduces interest in the policy, but neither influences support for the policy nor triggers general concerns about online misinformation.
  arXiv  Detail & Related papers  (2025-06-19T10:32:52Z)
• RAID: A Dataset for Testing the Adversarial Robustness of AI-Generated Image Detectors [57.81012948133832]
  We present RAID (Robust evaluation of AI-generated image Detectors), a dataset of 72k diverse and highly transferable adversarial examples.<n>Our methodology generates adversarial images that transfer with a high success rate to unseen detectors.<n>Our findings indicate that current state-of-the-art AI-generated image detectors can be easily deceived by adversarial examples.
  arXiv  Detail & Related papers  (2025-06-04T14:16:00Z)
• Adoption of Watermarking Measures for AI-Generated Content and Implications under the EU AI Act [4.2125200966193885]
  This paper provides an empirical analysis of 50 widely used AI systems for image generation, embedded into a legal analysis of the AI Act.<n>We find that only a minority of AI image generators currently implement adequate watermarking and deep fake labelling.
  arXiv  Detail & Related papers  (2025-03-23T17:55:33Z)
• Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing [55.2480439325792]
  This study systematically evaluations twelve state-of-the-art AI-text detectors using our AI-Polished-Text Evaluation dataset.<n>Our findings reveal that detectors frequently flag even minimally polished text as AI-generated, struggle to differentiate between degrees of AI involvement, and exhibit biases against older and smaller models.
  arXiv  Detail & Related papers  (2025-02-21T18:45:37Z)
• Labeling Synthetic Content: User Perceptions of Warning Label Designs for AI-generated Content on Social Media [16.5125333136211]
  We devised and assessed ten distinct label design samples that varied across the dimensions of sentiment, color/iconography, positioning, and level of detail.<n>Our experimental study involved 911 participants randomly assigned to these ten label designs and a control group evaluating social media content.<n>The results demonstrate that the presence of labels had a significant effect on the users belief that the content is AI generated, deepfake, or edited by AI.
  arXiv  Detail & Related papers  (2025-02-14T10:35:42Z)
• Detecting Discrepancies Between AI-Generated and Natural Images Using Uncertainty [91.64626435585643]
  We propose a novel approach for detecting AI-generated images by leveraging predictive uncertainty to mitigate misuse and associated risks.<n>The motivation arises from the fundamental assumption regarding the distributional discrepancy between natural and AI-generated images.<n>We propose to leverage large-scale pre-trained models to calculate the uncertainty as the score for detecting AI-generated images.
  arXiv  Detail & Related papers  (2024-12-08T11:32:25Z)
• Human Bias in the Face of AI: The Role of Human Judgement in AI Generated Text Evaluation [48.70176791365903]
  This study explores how bias shapes the perception of AI versus human generated content. We investigated how human raters respond to labeled and unlabeled content.
  arXiv  Detail & Related papers  (2024-09-29T04:31:45Z)
• DeepfakeArt Challenge: A Benchmark Dataset for Generative AI Art Forgery and Data Poisoning Detection [57.51313366337142]
  There has been growing concern over the use of generative AI for malicious purposes. In the realm of visual content synthesis using generative AI, key areas of significant concern has been image forgery and data poisoning. We introduce the DeepfakeArt Challenge, a large-scale challenge benchmark dataset designed specifically to aid in the building of machine learning algorithms for generative AI art forgery and data poisoning detection.
  arXiv  Detail & Related papers  (2023-06-02T05:11:27Z)
• Seeing is not always believing: Benchmarking Human and Model Perception of AI-Generated Images [66.20578637253831]
  There is a growing concern that the advancement of artificial intelligence (AI) technology may produce fake photos. This study aims to comprehensively evaluate agents for distinguishing state-of-the-art AI-generated visual content.
  arXiv  Detail & Related papers  (2023-04-25T17:51:59Z)
• Explainable AI for Natural Adversarial Images [4.387699521196243]
  Humans tend to assume that the AI's decision process mirrors their own. Here we evaluate if methods from explainable AI can disrupt this assumption to help participants predict AI classifications for adversarial and standard images. We find that both saliency maps and examples facilitate catching AI errors, but their effects are not additive, and saliency maps are more effective than examples.
  arXiv  Detail & Related papers  (2021-06-16T20:19:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.