Unmasking the Unknown: Facial Deepfake Detection in the Open-Set Paradigm

• URL: http://arxiv.org/abs/2503.08055v1
• Date: Tue, 11 Mar 2025 05:23:07 GMT
• Title: Unmasking the Unknown: Facial Deepfake Detection in the Open-Set Paradigm
• Authors: Nadarasar Bahavan, Sanjay Saha, Ken Chen, Sachith Seneviratne, Sanka Rasnayaka, Saman Halgamuge,
• Abstract summary: We propose a shift from the closed-set paradigm for deepfake detection.<n>In this paper, we propose an open-set deepfake classification algorithm based on supervised contrastive learning.
• Score: 4.505727709365421
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Facial forgery methods such as deepfakes can be misused for identity manipulation and spreading misinformation. They have evolved alongside advancements in generative AI, leading to new and more sophisticated forgery techniques that diverge from existing 'known' methods. Conventional deepfake detection methods use the closedset paradigm, thus limiting their applicability to detecting forgeries created using methods that are not part of the training dataset. In this paper, we propose a shift from the closed-set paradigm for deepfake detection. In the open-set paradigm, models are designed not only to identify images created by known facial forgery methods but also to identify and flag those produced by previously unknown methods as 'unknown' and not as unforged/real/unmanipulated. In this paper, we propose an open-set deepfake classification algorithm based on supervised contrastive learning. The open-set paradigm used in our model allows it to function as a more robust tool capable of handling emerging and unseen deepfake techniques, enhancing reliability and confidence, and complementing forensic analysis. In open-set paradigm, we identify three groups including the "unknown group that is neither considered known deepfake nor real. We investigate deepfake open-set classification across three scenarios, classifying deepfakes from unknown methods not as real, distinguishing real images from deepfakes, and classifying deepfakes from known methods, using the FaceForensics++ dataset as a benchmark. Our method achieves state of the art results in the first two tasks and competitive results in the third task.

Related papers

• DiffFake: Exposing Deepfakes using Differential Anomaly Detection [16.528373143163275]
  We propose DiffFake, a novel deepfake detector that approaches the detection problem as an anomaly detection task.<n>Specifically, DiffFake learns natural changes that occur between two facial images of the same person by leveraging a differential anomaly detection framework.<n>We show that our method can match and sometimes even exceed the performance of state-of-the-art competitors.
  arXiv  Detail & Related papers  (2025-02-22T14:50:53Z)
• Time-Aware Face Anti-Spoofing with Rotation Invariant Local Binary Patterns and Deep Learning [50.79277723970418]
  imitation attacks can lead to erroneous identification and subsequent authentication of attackers. Similar to face recognition, imitation attacks can also be detected with Machine Learning. We propose a novel approach that promises high classification accuracy by combining previously unused features with time-aware deep learning strategies.
  arXiv  Detail & Related papers  (2024-08-27T07:26:10Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• Deepfake Detection without Deepfakes: Generalization via Synthetic Frequency Patterns Injection [12.33030785907372]
  Deepfake detectors are typically trained on large sets of pristine and generated images. Deepfake detectors excel at identifying deepfakes created through methods encountered during training but struggle with those generated by unknown techniques. This paper introduces a learning approach aimed at significantly enhancing the generalization capabilities of deepfake detectors.
  arXiv  Detail & Related papers  (2024-03-20T10:33:10Z)
• Unsupervised Multimodal Deepfake Detection Using Intra- and Cross-Modal Inconsistencies [14.660707087391463]
  Deepfake videos present an increasing threat to society with potentially negative impact on criminal justice, democracy, and personal safety and privacy. We propose a novel unsupervised method for detecting deepfake videos by directly identifying intra-modal and cross-modal inconsistency between video segments. Our proposed method outperforms prior state-of-the-art unsupervised deepfake detection methods on the challenging FakeAVCeleb dataset.
  arXiv  Detail & Related papers  (2023-11-28T03:28:19Z)
• Facial Forgery-based Deepfake Detection using Fine-Grained Features [7.378937711027777]
  Facial forgery by deepfakes has caused major security risks and raised severe societal concerns. We formulate deepfake detection as a fine-grained classification problem and propose a new fine-grained solution to it. Our method is based on learning subtle and generalizable features by effectively suppressing background noise and learning discriminative features at various scales for deepfake detection.
  arXiv  Detail & Related papers  (2023-10-10T21:30:05Z)
• CrossDF: Improving Cross-Domain Deepfake Detection with Deep Information Decomposition [53.860796916196634]
  We propose a Deep Information Decomposition (DID) framework to enhance the performance of Cross-dataset Deepfake Detection (CrossDF) Unlike most existing deepfake detection methods, our framework prioritizes high-level semantic features over specific visual artifacts. It adaptively decomposes facial features into deepfake-related and irrelevant information, only using the intrinsic deepfake-related information for real/fake discrimination.
  arXiv  Detail & Related papers  (2023-09-30T12:30:25Z)
• The Familiarity Hypothesis: Explaining the Behavior of Deep Open Set Methods [86.39044549664189]
  Anomaly detection algorithms for feature-vector data identify anomalies as outliers, but outlier detection has not worked well in deep learning. This paper proposes the Familiarity Hypothesis that these methods succeed because they are detecting the absence of familiar learned features rather than the presence of novelty. The paper concludes with a discussion of whether familiarity detection is an inevitable consequence of representation learning.
  arXiv  Detail & Related papers  (2022-03-04T18:32:58Z)
• Understanding the Security of Deepfake Detection [23.118012417901078]
  We study the security of state-of-the-art deepfake detection methods in adversarial settings. We use two large-scale public deepfakes data sources including FaceForensics++ and Facebook Deepfake Detection Challenge. Our results uncover multiple security limitations of the deepfake detection methods in adversarial settings.
  arXiv  Detail & Related papers  (2021-07-05T14:18:21Z)
• Multi-attentional Deepfake Detection [79.80308897734491]
  Face forgery by deepfake is widely spread over the internet and has raised severe societal concerns. We propose a new multi-attentional deepfake detection network. Specifically, it consists of three key components: 1) multiple spatial attention heads to make the network attend to different local parts; 2) textural feature enhancement block to zoom in the subtle artifacts in shallow features; 3) aggregate the low-level textural feature and high-level semantic features guided by the attention maps.
  arXiv  Detail & Related papers  (2021-03-03T13:56:14Z)
• Identity-Driven DeepFake Detection [91.0504621868628]
  Identity-Driven DeepFake Detection takes as input the suspect image/video as well as the target identity information. We output a decision on whether the identity in the suspect image/video is the same as the target identity. We present a simple identity-based detection algorithm called the OuterFace, which may serve as a baseline for further research.
  arXiv  Detail & Related papers  (2020-12-07T18:59:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.