Testing Human Ability To Detect Deepfake Images of Human Faces

• URL: http://arxiv.org/abs/2212.05056v3
• Date: Thu, 25 May 2023 15:07:19 GMT
• Title: Testing Human Ability To Detect Deepfake Images of Human Faces
• Authors: Sergi D. Bray (1), Shane D. Johnson (1), Bennett Kleinberg (2) ((1) University College London, (2) Tilburg University)
• Abstract summary: In 2020 a workshop consulting AI experts ranked deepfakes as the most serious AI threat. This study aims to assess human ability to identify image deepfakes of human faces.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deepfakes are computationally-created entities that falsely represent reality. They can take image, video, and audio modalities, and pose a threat to many areas of systems and societies, comprising a topic of interest to various aspects of cybersecurity and cybersafety. In 2020 a workshop consulting AI experts from academia, policing, government, the private sector, and state security agencies ranked deepfakes as the most serious AI threat. These experts noted that since fake material can propagate through many uncontrolled routes, changes in citizen behaviour may be the only effective defence. This study aims to assess human ability to identify image deepfakes of human faces (StyleGAN2:FFHQ) from nondeepfake images (FFHQ), and to assess the effectiveness of simple interventions intended to improve detection accuracy. Using an online survey, 280 participants were randomly allocated to one of four groups: a control group, and 3 assistance interventions. Each participant was shown a sequence of 20 images randomly selected from a pool of 50 deepfake and 50 real images of human faces. Participants were asked if each image was AI-generated or not, to report their confidence, and to describe the reasoning behind each response. Overall detection accuracy was only just above chance and none of the interventions significantly improved this. Participants' confidence in their answers was high and unrelated to accuracy. Assessing the results on a per-image basis reveals participants consistently found certain images harder to label correctly, but reported similarly high confidence regardless of the image. Thus, although participant accuracy was 62% overall, this accuracy across images ranged quite evenly between 85% and 30%, with an accuracy of below 50% for one in every five images. We interpret the findings as suggesting that there is a need for an urgent call to action to address this threat.

Related papers

• We are not able to identify AI-generated images [0.0]
  Our dataset contains 120 difficult cases: real images sampled from CC12M, and carefully curated AI-generated counterparts produced with MidJourney.<n>Our results indicate that, even on relatively simple portrait images, humans struggle to reliably detect AI-generated content.<n>These findings highlight the need for greater awareness and ethical guidelines as AI-generated media becomes increasingly indistinguishable from reality.
  arXiv  Detail & Related papers  (2025-12-23T11:55:40Z)
• Bridging the Gap Between Ideal and Real-world Evaluation: Benchmarking AI-Generated Image Detection in Challenging Scenarios [54.07895223545793]
  This paper introduces the Real-World Robustness dataset (RRDataset) for comprehensive evaluation of detection models across three dimensions.<n>RRDataset includes high-quality images from seven major scenarios.<n>We benchmarked 17 detectors and 10 vision-language models (VLMs) on RRDataset and conducted a large-scale human study.
  arXiv  Detail & Related papers  (2025-09-11T06:15:52Z)
• Digital literacy interventions can boost humans in discerning deepfakes [20.57872238271025]
  Deepfakes, i.e., images generated by artificial intelligence (AI), can erode trust in institutions and compromise election outcomes.<n>Here, we compare the efficacy of five digital literacy interventions to boost people's ability to discern deepfakes.<n>Our results show that our interventions can boost deepfake discernment by up to 13 percentage points while maintaining trust in real images.
  arXiv  Detail & Related papers  (2025-07-31T12:23:45Z)
• Unmasking Illusions: Understanding Human Perception of Audiovisual Deepfakes [49.81915942821647]
  This paper aims to evaluate the human ability to discern deepfake videos through a subjective study. We present our findings by comparing human observers to five state-ofthe-art audiovisual deepfake detection models. We found that all AI models performed better than humans when evaluated on the same 40 videos.
  arXiv  Detail & Related papers  (2024-05-07T07:57:15Z)
• Turn Fake into Real: Adversarial Head Turn Attacks Against Deepfake Detection [58.1263969438364]
  We propose adversarial head turn (AdvHeat) as the first attempt at 3D adversarial face views against deepfake detectors. Experiments validate the vulnerability of various detectors to AdvHeat in realistic, black-box scenarios. Additional analyses demonstrate that AdvHeat is better than conventional attacks on both the cross-detector transferability and robustness to defenses.
  arXiv  Detail & Related papers  (2023-09-03T07:01:34Z)
• Seeing is not always believing: Benchmarking Human and Model Perception of AI-Generated Images [66.20578637253831]
  There is a growing concern that the advancement of artificial intelligence (AI) technology may produce fake photos. This study aims to comprehensively evaluate agents for distinguishing state-of-the-art AI-generated visual content.
  arXiv  Detail & Related papers  (2023-04-25T17:51:59Z)
• Analyzing Human Observer Ability in Morphing Attack Detection -- Where Do We Stand? [11.37940154420898]
  A prevalent misconception is that an examiner's or observer's capacity for facial morph detection depends on their subject expertise, experience, and familiarity with the issue. This study builds a new benchmark database of realistic morphing attacks from 48 different subjects, resulting in 400 morphed images. We also capture images from Automated Border Control (ABC) gates to mimic the realistic border-crossing scenarios in the D-MAD setting with 400 probe images to study the ability of human observers to detect morphed images.
  arXiv  Detail & Related papers  (2022-02-24T23:46:22Z)
• On the Effect of Selfie Beautification Filters on Face Detection and Recognition [53.561797148529664]
  Social media image filters modify the image contrast or illumination or occlude parts of the face with for example artificial glasses or animal noses. We develop a method to reconstruct the applied manipulation with a modified version of the U-NET segmentation network. From a recognition perspective, we employ distance measures and trained machine learning algorithms applied to features extracted using a ResNet-34 network trained to recognize faces.
  arXiv  Detail & Related papers  (2021-10-17T22:10:56Z)
• Dodging Attack Using Carefully Crafted Natural Makeup [42.65417043860506]
  We present a novel black-box adversarial machine learning (AML) attack which crafts natural makeup on a human participant. We evaluate our proposed attack against the ArcFace face recognition model, with 20 participants in a real-world setup. In the digital domain, the face recognition system was unable to identify all of the participants, while in the physical domain, the face recognition system was able to identify the participants in only 1.22% of the frames.
  arXiv  Detail & Related papers  (2021-09-14T06:27:14Z)
• DeeperForensics Challenge 2020 on Real-World Face Forgery Detection: Methods and Results [144.5252578415748]
  This paper reports methods and results in the DeeperForensics Challenge 2020 on real-world face forgery detection. The challenge employs the DeeperForensics-1.0 dataset, with 60,000 videos constituted by a total of 17.6 million frames. A total of 115 participants registered for the competition, and 25 teams made valid submissions.
  arXiv  Detail & Related papers  (2021-02-18T16:48:57Z)
• FaceGuard: A Self-Supervised Defense Against Adversarial Face Images [59.656264895721215]
  We propose a new self-supervised adversarial defense framework, namely FaceGuard, that can automatically detect, localize, and purify a wide variety of adversarial faces. During training, FaceGuard automatically synthesizes challenging and diverse adversarial attacks, enabling a classifier to learn to distinguish them from real faces. Experimental results on LFW dataset show that FaceGuard can achieve 99.81% detection accuracy on six unseen adversarial attack types.
  arXiv  Detail & Related papers  (2020-11-28T21:18:46Z)
• Fighting Deepfake by Exposing the Convolutional Traces on Images [0.0]
  Mobile apps like FACEAPP make use of the most advanced Generative Adversarial Networks (GAN) to produce extreme transformations on human face photos. This kind of media object took the name of Deepfake and raised a new challenge in the multimedia forensics field: the Deepfake detection challenge. In this paper, a new approach aimed to extract a Deepfake fingerprint from images is proposed.
  arXiv  Detail & Related papers  (2020-08-07T08:49:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.