SELFI: Selective Fusion of Identity for Generalizable Deepfake Detection

• URL: http://arxiv.org/abs/2506.17592v1
• Date: Sat, 21 Jun 2025 05:11:35 GMT
• Title: SELFI: Selective Fusion of Identity for Generalizable Deepfake Detection
• Abstract summary: Face identity provides a powerful signal for deepfake detection.<n>Some suppress identity cues to reduce bias, while others rely on them as forensic evidence.<n>We propose textbfSELFI (textbfSELective textbfIdentity), a generalizable detection framework that dynamically modulates identity usage.
• Score: 16.500269508552844
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Face identity provides a powerful signal for deepfake detection. Prior studies show that even when not explicitly modeled, classifiers often learn identity features implicitly. This has led to conflicting views: some suppress identity cues to reduce bias, while others rely on them as forensic evidence. To reconcile these views, we analyze two hypotheses: (1) whether face identity alone is discriminative for detecting deepfakes, and (2) whether such identity features generalize poorly across manipulation methods. Our experiments confirm that identity is informative but context-dependent. While some manipulations preserve identity-consistent artifacts, others distort identity cues and harm generalization. We argue that identity features should neither be blindly suppressed nor relied upon, but instead be explicitly modeled and adaptively controlled based on per-sample relevance. We propose \textbf{SELFI} (\textbf{SEL}ective \textbf{F}usion of \textbf{I}dentity), a generalizable detection framework that dynamically modulates identity usage. SELFI consists of: (1) a Forgery-Aware Identity Adapter (FAIA) that extracts identity embeddings from a frozen face recognition model and projects them into a forgery-relevant space via auxiliary supervision; and (2) an Identity-Aware Fusion Module (IAFM) that selectively integrates identity and visual features using a relevance-guided fusion mechanism. Experiments on four benchmarks show that SELFI improves cross-manipulation generalization, outperforming prior methods by an average of 3.1\% AUC. On the challenging DFDC dataset, SELFI exceeds the previous best by 6\%. Code will be released upon paper acceptance.

Related papers

• WithEveryone: Unified Planning and Identity Grounding for Group Image Generation [60.528070962852716]
  We introduce WithEveryone, a unified framework for generating group images up to ten reference identities.<n>WithEveryone injects each selected identity as an addressed token, predicts a structured identity--plan, and renders the plan as a visual condition.<n>It achieves the highest target-context identity similarity, improving face similarity from 0.462 for GPT-Image-2 to 0.499, while reducing copy-paste artifacts from 0.169 to 0.055.
  arXiv  Detail & Related papers  (2026-08-20T17:59:53Z)
• Diff-ID: Identity Consistent Facial Image Generation and Morphing via Diffusion Models [42.651660700496606]
  We introduce Diff-ID, a diffusion based framework that enforces identity consistency while delivering photorealistic quality.<n>We propose a pseudo discriminator loss based on ArcFace cosine similarity with exponential timestep weighting to reinforce identity perceptual fidelity.<n>Experiments show Diff-ID does not exceed InstantID in raw ArcFace Face Similarity, but achieves substantially lower FID and the strongest FIQ based identity--realism trade off among the evaluated methods.
  arXiv  Detail & Related papers  (2026-07-27T21:13:29Z)
• PIU: Proximity-guided Identity Unlearning in ID-Conditioned Diffusion Models [2.260041671242786]
  We study identity unlearning in Arc2Face, a state-of-the-art identity-conditioned latent diffusion model for face generation.<n>We introduce Proximity-guided Identity Unlearning (PIU), an anchor-guided framework for identity unlearning.
  arXiv  Detail & Related papers  (2026-05-21T10:55:56Z)
• AnyID: Ultra-Fidelity Universal Identity-Preserving Video Generation from Any Visual References [52.81991915377852]
  We present AnyID, an ultra-fidelity identity-preservation video generation framework.<n>First, we introduce a scalable omni-referenced architecture that effectively unifies heterogeneous identity inputs into a cohesive representation.<n>Second, we propose a primary-referenced generation paradigm, which designates one reference as a canonical anchor and uses a novel differential prompt to enable precise, attribute-level controllability.
  arXiv  Detail & Related papers  (2026-03-26T08:54:48Z)
• ExposeAnyone: Personalized Audio-to-Expression Diffusion Models Are Robust Zero-Shot Face Forgery Detectors [58.45131932883374]
  We propose a fully self-supervised approach to detect deepfakes in videos.<n>Our model computes the identity distances between suspected videos and personalized subjects via diffusion reconstruction errors.<n>Our method is highly robust to corruptions such as blur and compression, highlighting the applicability in real-world face forgery detection.
  arXiv  Detail & Related papers  (2026-01-05T18:59:54Z)
• BeyondFacial: Identity-Preserving Personalized Generation Beyond Facial Close-ups [22.017690133402912]
  Identity-Preserving Personalized Generation (I) has advanced film production and artistic creation, yet existing approaches overemphasize facial regions.<n>These methods suffer from weak visual narrativity and poor semantic consistency under complex text prompts.<n>This paper presents an I method that breaks the constraint of facial close-ups, achieving synergistic optimization of identity fidelity and scene semantic creation.
  arXiv  Detail & Related papers  (2025-11-15T01:56:14Z)
• Beyond Inference Intervention: Identity-Decoupled Diffusion for Face Anonymization [55.29071072675132]
  Face anonymization aims to conceal identity information while preserving non-identity attributes.<n>We propose textbfIDsuperscript2Face, a training-centric anonymization framework.<n>We show that IDtextsuperscript2Face outperforms existing methods in visual quality, identity suppression, and utility preservation.
  arXiv  Detail & Related papers  (2025-10-28T09:28:12Z)
• Hybrid Generative Fusion for Efficient and Privacy-Preserving Face Recognition Dataset Generation [87.48785461212556]
  We present our approach to the DataCV ICCV Challenge, which centers on building a high-quality face dataset to train a face recognition model.<n>The constructed dataset must not contain identities overlapping with any existing public face datasets.<n>Our method achieves textbf1st place in the competition, and experimental results show that our dataset improves model performance across 10K, 20K, and 100K identity scales.
  arXiv  Detail & Related papers  (2025-08-14T14:14:18Z)
• FakeIDet: Exploring Patches for Privacy-Preserving Fake ID Detection [12.969417519807322]
  This study focuses on the topic of fake ID detection, covering several limitations in the field.<n>There are no publicly available data from real IDs for proper research in this area.<n>Most published studies rely on proprietary internal databases that are not available for privacy reasons.
  arXiv  Detail & Related papers  (2025-04-10T14:01:22Z)
• From Poses to Identity: Training-Free Person Re-Identification via Feature Centralization [9.614305363044737]
  Person re-identification (ReID) aims to extract accurate identity representation features.<n>We propose a Training-Free Feature Centralization ReID framework (Pose2ID) to reduce individual noise.<n>Our method sets new state-of-the-art results across standard, cross-modality, and occluded ReID tasks.
  arXiv  Detail & Related papers  (2025-03-02T15:31:48Z)
• ID$^3$: Identity-Preserving-yet-Diversified Diffusion Models for Synthetic Face Recognition [60.15830516741776]
  Synthetic face recognition (SFR) aims to generate datasets that mimic the distribution of real face data. We introduce a diffusion-fueled SFR model termed $textID3$. $textID3$ employs an ID-preserving loss to generate diverse yet identity-consistent facial appearances.
  arXiv  Detail & Related papers  (2024-09-26T06:46:40Z)
• Disentangled Representations for Short-Term and Long-Term Person Re-Identification [33.76874948187976]
  We propose a new generative adversarial network, dubbed identity shuffle GAN (IS-GAN) It disentangles identity-related and unrelated features from person images through an identity-shuffling technique. Experimental results validate the effectiveness of IS-GAN, showing state-of-the-art performance on standard reID benchmarks.
  arXiv  Detail & Related papers  (2024-09-09T02:09:49Z)
• ID-Aligner: Enhancing Identity-Preserving Text-to-Image Generation with Reward Feedback Learning [57.91881829308395]
  Identity-preserving text-to-image generation (ID-T2I) has received significant attention due to its wide range of application scenarios like AI portrait and advertising. We present textbfID-Aligner, a general feedback learning framework to enhance ID-T2I performance.
  arXiv  Detail & Related papers  (2024-04-23T18:41:56Z)
• Beyond Inserting: Learning Identity Embedding for Semantic-Fidelity Personalized Diffusion Generation [21.739328335601716]
  This paper focuses on inserting accurate and interactive ID embedding into the Stable Diffusion Model for personalized generation. We propose a face-wise attention loss to fit the face region instead of entangling ID-unrelated information, such as face layout and background. Our results exhibit superior ID accuracy, text-based manipulation ability, and generalization compared to previous methods.
  arXiv  Detail & Related papers  (2024-01-31T11:52:33Z)
• Disguise without Disruption: Utility-Preserving Face De-Identification [40.484745636190034]
  We introduce Disguise, a novel algorithm that seamlessly de-identifies facial images while ensuring the usability of the modified data. Our method involves extracting and substituting depicted identities with synthetic ones, generated using variational mechanisms to maximize obfuscation and non-invertibility. We extensively evaluate our method using multiple datasets, demonstrating a higher de-identification rate and superior consistency compared to prior approaches in various downstream tasks.
  arXiv  Detail & Related papers  (2023-03-23T13:50:46Z)
• Dynamic Prototype Mask for Occluded Person Re-Identification [88.7782299372656]
  Existing methods mainly address this issue by employing body clues provided by an extra network to distinguish the visible part. We propose a novel Dynamic Prototype Mask (DPM) based on two self-evident prior knowledge. Under this condition, the occluded representation could be well aligned in a selected subspace spontaneously.
  arXiv  Detail & Related papers  (2022-07-19T03:31:13Z)
• Identity Documents Authentication based on Forgery Detection of Guilloche Pattern [2.606834301724095]
  An authentication model for identity documents based on forgery detection of guilloche patterns is proposed. Experiments are conducted in order to analyze and identify the most proper parameters to achieve higher authentication performance.
  arXiv  Detail & Related papers  (2022-06-22T11:37:10Z)
• Protecting Celebrities with Identity Consistency Transformer [119.67996461810304]
  Identity Consistency Transformer focuses on high-level semantics, specifically identity information, and detecting a suspect face by finding identity inconsistency in inner and outer face regions. We show that Identity Consistency Transformer exhibits superior generalization ability not only across different datasets but also across various types of image degradation forms found in real-world applications including deepfake videos.
  arXiv  Detail & Related papers  (2022-03-02T18:59:58Z)
• Inductive Biased Estimation: Learning Generalizations for Identity Transfer [64.4487809928537]
  This paper proposes an Errors-in-Variables Adapter (EVA) model to induce learning of proper generalizations. To better match the source face with the target situation in terms of pose, expression, and background factors, we model the bias as a causal effect of the target situation on source identity.
  arXiv  Detail & Related papers  (2021-10-04T17:10:30Z)
• Identity-Driven DeepFake Detection [91.0504621868628]
  Identity-Driven DeepFake Detection takes as input the suspect image/video as well as the target identity information. We output a decision on whether the identity in the suspect image/video is the same as the target identity. We present a simple identity-based detection algorithm called the OuterFace, which may serve as a baseline for further research.
  arXiv  Detail & Related papers  (2020-12-07T18:59:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.