Beyond Ambiguous Visual Cues: Studying Physiological Disruptions and Cross-Modal Inconsistencies in Deepfake Videos

• URL: http://arxiv.org/abs/2609.12668v1
• Date: Fri, 11 Sep 2026 10:12:48 GMT
• Title: Beyond Ambiguous Visual Cues: Studying Physiological Disruptions and Cross-Modal Inconsistencies in Deepfake Videos
• Abstract summary: Current detectors underexplore the cross-level relationship between facial features and physiological dynamics, often relying on late fusion or r features alone.<n>We propose a bidirectional co-attention fusion detector that captures the cross-level dependencies between pulse dynamics and facial motion to learn a robust, joint authenticity representation.
• Score: 7.77578121448493
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent deepfake detection studies increasingly suggest remote photoplethysmography (rPPG) signals as an authenticity cue. However, existing benchmarks lack physiological ground truth, and current detectors underexplore the cross-level relationship between facial features and physiological dynamics, often relying on late fusion or rPPG features alone. In this paper, we construct high-fidelity deepfake manipulations on established real rPPG datasets (COHFACE and UBFC-rPPG) to investigate how forgeries disrupt natural physiological signals and facial behavior at the same time. Building on this analysis, we propose a bidirectional co-attention fusion detector that jointly models rPPG and facial behavior tokens. This mechanism explicitly captures the cross-level dependencies between pulse dynamics and facial motion to learn a robust, joint authenticity representation. Extensive experiments using a subject-disjoint 5-fold evaluation demonstrate the superiority of our approach. Achieving a 92.80\% AUC on constructed datasets using face swapping and 96.78\% AUC on motion transfer, our model outperforms both the rPPG-only single modality baseline and the best feature-level fusion methods. Furthermore, transfer-learning result of the fusion detector on Celeb-DF-v2 while keeping both feature extractors fixed achieves 91.20\% accuracy and 86.08\% AUC, which suggests applicability under target-domain adaptation.

Related papers

• Physiological Signals as a Forensic Modality for Talking-Face Deepfake Detection [0.0]
  Talking-face (TF) deep-swap generation synthesizes photore- alistic facial video from a static source image and an au-fake signal, producing forgeries that current image-based detectors consistently fail to identify.<n>We propose a detection framework that extracts per-video r wave forms via RhythmFormer and trains a suite of lightweight classifiers to distinguish real signals.
  arXiv  Detail & Related papers  (2026-07-23T19:47:15Z)
• Intervention-Based Self-Supervised Learning: A Causal Probe Paradigm for Remote Photoplethysmography [38.62089038790769]
  We propose a new SSL paradigm, Causal Probing (PCP)<n>PCP treats the latent r signal as the underlying physical source and the resulting pixel chrominance variations as its visual manifestation.<n>Our editor achieves both in-domain and cross-domain performance on challenging datasets such as VIPL-HR and MMPD-HR.
  arXiv  Detail & Related papers  (2026-04-26T09:56:09Z)
• Face-D(^2)CL: Multi-Domain Synergistic Representation with Dual Continual Learning for Facial DeepFake Detection [21.09692782202372]
  Continual learning provides an effective approach to adapt facial DeepFake detection models to evolving forgery patterns.<n>We propose Face-D(2)CL, a framework for facial DeepFake detection.
  arXiv  Detail & Related papers  (2026-04-09T12:18:42Z)
• Function-Space Decoupled Diffusion for Forward and Inverse Modeling in Carbon Capture and Storage [65.51149575007149]
  We present Fun-DDPS, a generative framework that combines function-space diffusion models with differentiable neural operator surrogates for both forward and inverse modeling.<n>Fun-DDPS produces physically consistent realizations free from the high-frequency artifacts observed in joint-state baselines.
  arXiv  Detail & Related papers  (2026-02-12T18:58:12Z)
• Reperio-rPPG: Relational Temporal Graph Neural Networks for Periodicity Learning in Remote Physiological Measurement [0.0]
  Remote photoplethys (rmography) is an emerging physiological sensing technique that leverages subtle color variations in facial videos to estimate vital signs such as heart rate and respiratory rate.<n>This non-invasive technique has gained traction across diverse domains, but its ability to capture fine-grained temporal dynamics under real-world conditions has been underexplored.<n>We propose Graph Reperio-r, a novel framework that strategically integrates a Transformer to effectively capture the periodic structure.
  arXiv  Detail & Related papers  (2025-11-08T09:41:34Z)
• AdaFusion: Prompt-Guided Inference with Adaptive Fusion of Pathology Foundation Models [49.550545038402184]
  We propose AdaFusion, a novel prompt-guided inference framework.<n>Our method compresses and aligns tile-level features from diverse models.<n>AdaFusion consistently surpasses individual PFMs across both classification and regression tasks.
  arXiv  Detail & Related papers  (2025-08-07T07:09:31Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• Benchmarking Joint Face Spoofing and Forgery Detection with Visual and Physiological Cues [81.15465149555864]
  We establish the first joint face spoofing and detection benchmark using both visual appearance and physiological r cues. To enhance the r periodicity discrimination, we design a two-branch physiological network using both facial powerful rtemporal signal map and its continuous wavelet transformed counterpart as inputs.
  arXiv  Detail & Related papers  (2022-08-10T15:41:48Z)
• Consistency Regularization for Deep Face Anti-Spoofing [69.70647782777051]
  Face anti-spoofing (FAS) plays a crucial role in securing face recognition systems. Motivated by this exciting observation, we conjecture that encouraging feature consistency of different views may be a promising way to boost FAS models. We enhance both Embedding-level and Prediction-level Consistency Regularization (EPCR) in FAS.
  arXiv  Detail & Related papers  (2021-11-24T08:03:48Z)
• Real-time landmark detection for precise endoscopic submucosal dissection via shape-aware relation network [51.44506007844284]
  We propose a shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection surgery. We first devise an algorithm to automatically generate relation keypoint heatmaps, which intuitively represent the prior knowledge of spatial relations among landmarks. We then develop two complementary regularization schemes to progressively incorporate the prior knowledge into the training process.
  arXiv  Detail & Related papers  (2021-11-08T07:57:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.