SELFI: Selective Fusion of Identity for Generalizable Deepfake Detection

• URL: http://arxiv.org/abs/2506.17592v1
• Date: Sat, 21 Jun 2025 05:11:35 GMT
• Title: SELFI: Selective Fusion of Identity for Generalizable Deepfake Detection
• Authors: Younghun Kim, Minsuk Jang, Myung-Joon Kwon, Wonjun Lee, Changick Kim,
• Abstract summary: Face identity provides a powerful signal for deepfake detection.<n>Some suppress identity cues to reduce bias, while others rely on them as forensic evidence.<n>We propose textbfSELFI (textbfSELective textbfIdentity), a generalizable detection framework that dynamically modulates identity usage.
• Score: 16.500269508552844
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Face identity provides a powerful signal for deepfake detection. Prior studies show that even when not explicitly modeled, classifiers often learn identity features implicitly. This has led to conflicting views: some suppress identity cues to reduce bias, while others rely on them as forensic evidence. To reconcile these views, we analyze two hypotheses: (1) whether face identity alone is discriminative for detecting deepfakes, and (2) whether such identity features generalize poorly across manipulation methods. Our experiments confirm that identity is informative but context-dependent. While some manipulations preserve identity-consistent artifacts, others distort identity cues and harm generalization. We argue that identity features should neither be blindly suppressed nor relied upon, but instead be explicitly modeled and adaptively controlled based on per-sample relevance. We propose \textbf{SELFI} (\textbf{SEL}ective \textbf{F}usion of \textbf{I}dentity), a generalizable detection framework that dynamically modulates identity usage. SELFI consists of: (1) a Forgery-Aware Identity Adapter (FAIA) that extracts identity embeddings from a frozen face recognition model and projects them into a forgery-relevant space via auxiliary supervision; and (2) an Identity-Aware Fusion Module (IAFM) that selectively integrates identity and visual features using a relevance-guided fusion mechanism. Experiments on four benchmarks show that SELFI improves cross-manipulation generalization, outperforming prior methods by an average of 3.1\% AUC. On the challenging DFDC dataset, SELFI exceeds the previous best by 6\%. Code will be released upon paper acceptance.

Related papers

• FakeIDet: Exploring Patches for Privacy-Preserving Fake ID Detection [12.969417519807322]
  This study focuses on the topic of fake ID detection, covering several limitations in the field.<n>There are no publicly available data from real IDs for proper research in this area.<n>Most published studies rely on proprietary internal databases that are not available for privacy reasons.
  arXiv  Detail & Related papers  (2025-04-10T14:01:22Z)
• From Poses to Identity: Training-Free Person Re-Identification via Feature Centralization [9.614305363044737]
  Person re-identification (ReID) aims to extract accurate identity representation features.<n>We propose a Training-Free Feature Centralization ReID framework (Pose2ID) to reduce individual noise.<n>Our method sets new state-of-the-art results across standard, cross-modality, and occluded ReID tasks.
  arXiv  Detail & Related papers  (2025-03-02T15:31:48Z)
• ID$^3$: Identity-Preserving-yet-Diversified Diffusion Models for Synthetic Face Recognition [60.15830516741776]
  Synthetic face recognition (SFR) aims to generate datasets that mimic the distribution of real face data. We introduce a diffusion-fueled SFR model termed $textID3$. $textID3$ employs an ID-preserving loss to generate diverse yet identity-consistent facial appearances.
  arXiv  Detail & Related papers  (2024-09-26T06:46:40Z)
• Disentangled Representations for Short-Term and Long-Term Person Re-Identification [33.76874948187976]
  We propose a new generative adversarial network, dubbed identity shuffle GAN (IS-GAN) It disentangles identity-related and unrelated features from person images through an identity-shuffling technique. Experimental results validate the effectiveness of IS-GAN, showing state-of-the-art performance on standard reID benchmarks.
  arXiv  Detail & Related papers  (2024-09-09T02:09:49Z)
• ID-Aligner: Enhancing Identity-Preserving Text-to-Image Generation with Reward Feedback Learning [57.91881829308395]
  Identity-preserving text-to-image generation (ID-T2I) has received significant attention due to its wide range of application scenarios like AI portrait and advertising. We present textbfID-Aligner, a general feedback learning framework to enhance ID-T2I performance.
  arXiv  Detail & Related papers  (2024-04-23T18:41:56Z)
• Beyond Inserting: Learning Identity Embedding for Semantic-Fidelity Personalized Diffusion Generation [21.739328335601716]
  This paper focuses on inserting accurate and interactive ID embedding into the Stable Diffusion Model for personalized generation. We propose a face-wise attention loss to fit the face region instead of entangling ID-unrelated information, such as face layout and background. Our results exhibit superior ID accuracy, text-based manipulation ability, and generalization compared to previous methods.
  arXiv  Detail & Related papers  (2024-01-31T11:52:33Z)
• Disguise without Disruption: Utility-Preserving Face De-Identification [40.484745636190034]
  We introduce Disguise, a novel algorithm that seamlessly de-identifies facial images while ensuring the usability of the modified data. Our method involves extracting and substituting depicted identities with synthetic ones, generated using variational mechanisms to maximize obfuscation and non-invertibility. We extensively evaluate our method using multiple datasets, demonstrating a higher de-identification rate and superior consistency compared to prior approaches in various downstream tasks.
  arXiv  Detail & Related papers  (2023-03-23T13:50:46Z)
• Dynamic Prototype Mask for Occluded Person Re-Identification [88.7782299372656]
  Existing methods mainly address this issue by employing body clues provided by an extra network to distinguish the visible part. We propose a novel Dynamic Prototype Mask (DPM) based on two self-evident prior knowledge. Under this condition, the occluded representation could be well aligned in a selected subspace spontaneously.
  arXiv  Detail & Related papers  (2022-07-19T03:31:13Z)
• Identity Documents Authentication based on Forgery Detection of Guilloche Pattern [2.606834301724095]
  An authentication model for identity documents based on forgery detection of guilloche patterns is proposed. Experiments are conducted in order to analyze and identify the most proper parameters to achieve higher authentication performance.
  arXiv  Detail & Related papers  (2022-06-22T11:37:10Z)
• Protecting Celebrities with Identity Consistency Transformer [119.67996461810304]
  Identity Consistency Transformer focuses on high-level semantics, specifically identity information, and detecting a suspect face by finding identity inconsistency in inner and outer face regions. We show that Identity Consistency Transformer exhibits superior generalization ability not only across different datasets but also across various types of image degradation forms found in real-world applications including deepfake videos.
  arXiv  Detail & Related papers  (2022-03-02T18:59:58Z)
• Inductive Biased Estimation: Learning Generalizations for Identity Transfer [64.4487809928537]
  This paper proposes an Errors-in-Variables Adapter (EVA) model to induce learning of proper generalizations. To better match the source face with the target situation in terms of pose, expression, and background factors, we model the bias as a causal effect of the target situation on source identity.
  arXiv  Detail & Related papers  (2021-10-04T17:10:30Z)
• Identity-Driven DeepFake Detection [91.0504621868628]
  Identity-Driven DeepFake Detection takes as input the suspect image/video as well as the target identity information. We output a decision on whether the identity in the suspect image/video is the same as the target identity. We present a simple identity-based detection algorithm called the OuterFace, which may serve as a baseline for further research.
  arXiv  Detail & Related papers  (2020-12-07T18:59:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.