PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysis

• URL: http://arxiv.org/abs/2510.14241v1
• Date: Thu, 16 Oct 2025 02:51:42 GMT
• Title: PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysis
• Abstract summary: Phoneme-Temporal and Identity-Dynamic Analysis is a novel framework for deepfake detection.<n>It incorporates language, dynamic face motion, and facial identification cues to address limitations.<n>It significantly improves the detection of subtle deepfake alterations.
• Score: 22.49571926071395
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rise of manipulated media has made deepfakes a particularly insidious threat, involving various generative manipulations such as lip-sync modifications, face-swaps, and avatar-driven facial synthesis. Conventional detection methods, which predominantly depend on manually designed phoneme-viseme alignment thresholds, fundamental frame-level consistency checks, or a unimodal detection strategy, inadequately identify modern-day deepfakes generated by advanced generative models such as GANs, diffusion models, and neural rendering techniques. These advanced techniques generate nearly perfect individual frames yet inadvertently create minor temporal discrepancies frequently overlooked by traditional detectors. We present a novel multimodal audio-visual framework, Phoneme-Temporal and Identity-Dynamic Analysis(PIA), incorporating language, dynamic face motion, and facial identification cues to address these limitations. We utilize phoneme sequences, lip geometry data, and advanced facial identity embeddings. This integrated method significantly improves the detection of subtle deepfake alterations by identifying inconsistencies across multiple complementary modalities. Code is available at https://github.com/skrantidatta/PIA

Related papers

• From Talking to Singing: A New Challenge for Audio-Visual Deepfake Detection [23.76387977850747]
  Existing methods for audio-visual deepfake detection rely on cross-modal inconsistencies.<n>In singing, rhythmic vocalization weakens this coupling and introduces a nontrivial domain shift.<n>We construct the Singing Head DeepFake dataset using rhythm-aware generative models to fill the gap in singing benchmarks.
  arXiv  Detail & Related papers  (2026-05-27T04:33:39Z)
• Attribution-Guided Multimodal Deepfake Detection via Cross-Modal Forensic Fingerprints [4.598350938853635]
  We argue that a detector incapable of identifying how a video was forged is likely learning the wrong signal.<n>Unlike binary detection, attribution-guided learning imposes a stronger geometric constraint on the shared embedding space.<n>We propose the Attribution-Guided Multimodal Deepfake Detection framework, which jointly learns to detect and attribute manipulation.
  arXiv  Detail & Related papers  (2026-04-29T09:11:13Z)
• Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detection [23.48106270102081]
  This paper tackles the challenge of detecting partially manipulated facial deepfakes.<n>We leverage the Contrastive Language-Image Pre-training (CLIP) model, specifically its ViT-L/14 visual encoder.<n>The proposed approach utilizes parameter-efficient fine-tuning (PEFT) techniques, such as LN-tuning, to adjust a small subset of the model's parameters.
  arXiv  Detail & Related papers  (2025-03-25T14:10:54Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• The Tug-of-War Between Deepfake Generation and Detection [4.62070292702111]
  Multimodal generative models are rapidly evolving, leading to a surge in the generation of realistic video and audio. Deepfake videos, which can convincingly impersonate individuals, have particularly garnered attention due to their potential misuse. This survey paper examines the dual landscape of deepfake video generation and detection, emphasizing the need for effective countermeasures.
  arXiv  Detail & Related papers  (2024-07-08T17:49:41Z)
• Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges [40.11614155244292]
  This survey traces the evolution of deepfake detection from early single-modal methods to sophisticated multi-modal approaches.<n>We present a structured taxonomy of detection techniques and analyze the transition from GAN-based to diffusion model-driven deepfakes.
  arXiv  Detail & Related papers  (2024-06-11T05:48:04Z)
• DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection [67.3143177137102]
  Deepfake detection refers to detecting artificially generated or edited faces in images or videos. We propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces.
  arXiv  Detail & Related papers  (2023-12-07T07:19:45Z)
• Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
  Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust. Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model. We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
  arXiv  Detail & Related papers  (2023-07-31T10:22:33Z)
• MMNet: Multi-Collaboration and Multi-Supervision Network for Sequential Deepfake Detection [81.59191603867586]
  Sequential deepfake detection aims to identify forged facial regions with the correct sequence for recovery. The recovery of forged images requires knowledge of the manipulation model to implement inverse transformations. We propose Multi-Collaboration and Multi-Supervision Network (MMNet) that handles various spatial scales and sequential permutations in forged face images.
  arXiv  Detail & Related papers  (2023-07-06T02:32:08Z)
• NPVForensics: Jointing Non-critical Phonemes and Visemes for Deepfake Detection [50.33525966541906]
  Existing multimodal detection methods capture audio-visual inconsistencies to expose Deepfake videos. We propose a novel Deepfake detection method to mine the correlation between Non-critical Phonemes and Visemes, termed NPVForensics. Our model can be easily adapted to the downstream Deepfake datasets with fine-tuning.
  arXiv  Detail & Related papers  (2023-06-12T06:06:05Z)
• Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis [69.09526348527203]
  Deep generative models have led to highly realistic media, known as deepfakes, that are commonly indistinguishable from real to human eyes. We propose a novel fake detection that is designed to re-synthesize testing images and extract visual cues for detection. We demonstrate the improved effectiveness, cross-GAN generalization, and robustness against perturbations of our approach in a variety of detection scenarios.
  arXiv  Detail & Related papers  (2021-05-29T21:22:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.