GC-ConsFlow: Leveraging Optical Flow Residuals and Global Context for Robust Deepfake Detection

• URL: http://arxiv.org/abs/2501.13435v1
• Date: Thu, 23 Jan 2025 07:43:56 GMT
• Title: GC-ConsFlow: Leveraging Optical Flow Residuals and Global Context for Robust Deepfake Detection
• Abstract summary: Deepfake technology has enabled the generation of highly realistic manipulated videos, posing severe social and ethical challenges.<n>Existing Deepfake detection methods primarily focused on either spatial or temporal inconsistencies, neglecting the interplay between the two.<n>We propose the global context flow (GC-ConsFlow), a novel dual-stream framework that effectively integrates spatial and temporal features for robust Deepfake detection.
• Score: 23.05634856249282
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid development of Deepfake technology has enabled the generation of highly realistic manipulated videos, posing severe social and ethical challenges. Existing Deepfake detection methods primarily focused on either spatial or temporal inconsistencies, often neglecting the interplay between the two or suffering from interference caused by natural facial motions. To address these challenges, we propose the global context consistency flow (GC-ConsFlow), a novel dual-stream framework that effectively integrates spatial and temporal features for robust Deepfake detection. The global grouped context aggregation module (GGCA), integrated into the global context-aware frame flow stream (GCAF), enhances spatial feature extraction by aggregating grouped global context information, enabling the detection of subtle, spatial artifacts within frames. The flow-gradient temporal consistency stream (FGTC), rather than directly modeling the residuals, it is used to improve the robustness of temporal feature extraction against the inconsistency introduced by unnatural facial motion using optical flow residuals and gradient-based features. By combining these two streams, GC-ConsFlow demonstrates the effectiveness and robustness in capturing complementary spatiotemporal forgery traces. Extensive experiments show that GC-ConsFlow outperforms existing state-of-the-art methods in detecting Deepfake videos under various compression scenarios.

Related papers

• Making Every Step Count: Spatio-Temporal Information Allocation for Imaging Inverse Problems [63.904793414062816]
  We introduce two complementary and training-free components, i.e., Spectrum-based Scheduling (SAS) and Measurement-Prioritized Attention (MPA)<n>SAS distributes the available NFEs over flow time according to the spectrum and logSNR geometry, thus better balancing semantic exploration and detail refinement.<n>MPA guides information toward weakly constrained regions, thereby enhancing semantic and structural fidelity.
  arXiv  Detail & Related papers  (2026-08-12T07:39:22Z)
• FlowDec: Temporal Conditional Flow Decorruptor for Robust Continuous Vision-Language Navigation [8.00310534826872]
  Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to follow natural-language instructions in unseen scenes.<n>We introduce Temporal Flow Decorruptor (FlowDec), a novel image restoration framework tailored for LM-based VLN-CE.<n>FlowDec outperforms state-of-the-art decorruption methods in both navigation accuracy and generation latency.
  arXiv  Detail & Related papers  (2026-06-21T10:24:16Z)
• LC-Flow: Learning Local Continuous Optical Flow and Confidence from events [5.146238489510258]
  Event cameras capture brightness changes asynchronously with microsecond resolution, yet existing optical flow methods fail to exploit this temporal continuity.<n>We propose textbfLC-Flow, the first temporally continuous, learning-based optical flow estimator that operates purely from local events.
  arXiv  Detail & Related papers  (2026-05-23T14:42:24Z)
• Hybrid Congestion Classification Framework Using Flow-Guided Attention and Empirical Mode Decomposition [6.830580384536778]
  This study proposes FLO-EMD, a hybrid approach that couples motion-guided attention with empirical, data-driven temporal decomposition.<n>Experiments on 1,050 five-second clips from four surveillance networks show FLO-EMD 97.5% overall test accuracy.
  arXiv  Detail & Related papers  (2026-05-06T10:59:32Z)
• Robust Deepfake Detection: Mitigating Spatial Attention Drift via Calibrated Complementary Ensembles [8.588817718552418]
  Current deepfake detection models suffer from spatial attention drift under real-world compound degradations.<n>We propose a foundation-driven forensic framework that integrates an extreme compound degradation engine with a structurally constrained, multi-stream architecture.<n>Our approach yields highly stable zero-shot generalization, achieving Fourth Place in the NTIRE 2026 Robust Deepfake Detection Challenge at CVPR.
  arXiv  Detail & Related papers  (2026-04-28T17:32:48Z)
• RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow [51.43025173196566]
  Monocular scene flow estimation aims to recover dense 3D motion from image sequences.<n>We propose RAFT-MSF++, a self-supervised multi-frame framework that fuses temporal features to jointly estimate depth and scene flow.<n>Experiments show that RAFT-MSF++ achieves 24.14% SF-all on the KITTI Scene Flow benchmark, with a 30.99% improvement over the baseline and better robustness in occluded regions.
  arXiv  Detail & Related papers  (2026-04-21T11:32:49Z)
• Multi-modal Deepfake Detection and Localization with FPN-Transformer [21.022230340898556]
  We introduce a multi-modal deepfake detection and localization framework based on a Feature Pyramid-Transformer (FPN-Transformer)<n>A multi-scale feature pyramid is constructed through R-TLM blocks with localized attention mechanisms, enabling joint analysis of cross-context temporal dependencies.<n>We evaluate our approach on the test set of the IJCAI'25 DDL-AV benchmark, showing a good performance with a final score of 0.7535.
  arXiv  Detail & Related papers  (2025-11-11T09:33:39Z)
• DeepShield: Fortifying Deepfake Video Detection with Local and Global Forgery Analysis [59.8324489002129]
  We introduce DeepShield, a deepfake detection framework that balances local sensitivity and global generalization to improve robustness across unseen forgeries.<n>DeepShield appliestemporal artifact modeling and patch-wise supervision to capture fine-grained inconsistencies often overlooked by global models.
  arXiv  Detail & Related papers  (2025-10-29T07:35:29Z)
• Space Object Detection using Multi-frame Temporal Trajectory Completion Method [9.408807475654068]
  Space objects in Geostationary Earth Orbit (GEO) present significant detection challenges in optical imaging.<n>We enhance high-frequency features of GEO targets while suppressing background noise at the single-frame level through wavelet transform.<n>To effectively mitigate missing and false detections, a series of key steps including temporal matching and completion, temporal-consistency-based noise filtering, and progressive trajectory refinement are designed.
  arXiv  Detail & Related papers  (2025-10-22T04:04:27Z)
• RainDiff: End-to-end Precipitation Nowcasting Via Token-wise Attention Diffusion [64.49056527678606]
  We propose a Token-wise Attention integrated into not only the U-Net diffusion model but also the radar-temporal encoder.<n>Unlike prior approaches, our method integrates attention into the architecture without incurring the high resource cost typical of pixel-space diffusion.<n>Our experiments and evaluations demonstrate that the proposed method significantly outperforms state-of-the-art approaches, robustness local fidelity, generalization, and superior in complex precipitation forecasting scenarios.
  arXiv  Detail & Related papers  (2025-10-16T17:59:13Z)
• Towards Robust and Generalizable Continuous Space-Time Video Super-Resolution with Events [71.2439653098351]
  Continuous space-time video super-STVSR has garnered increasing interest for its capability to reconstruct high-resolution and high-frame-rate videos at arbitrary temporal scales.<n>We present EvEnhancer, a novel approach that marries unique properties of high temporal and high dynamic range encapsulated in event streams.<n>Our method achieves state-of-the-art performance on both synthetic and real-world datasets, while maintaining generalizability at OOD scales.
  arXiv  Detail & Related papers  (2025-10-04T15:23:07Z)
• Video Forgery Detection with Optical Flow Residuals and Spatial-Temporal Consistency [1.7061868168035932]
  We propose a detection framework that leverages spatial-temporal consistency by combining RGB appearance features with optical flow residuals.<n>By integrating these complementary features, the proposed method effectively detects a wide range of forged videos.
  arXiv  Detail & Related papers  (2025-08-01T07:51:35Z)
• GTAD: Global Temporal Aggregation Denoising Learning for 3D Semantic Occupancy Prediction [14.549066678968368]
  We propose a global temporal aggregation denoising network named GTAD for holistic 3D scene understanding.<n>Our method employs an in-model latent denoising network to aggregate local temporal features from the current moment and global temporal features from historical sequences.
  arXiv  Detail & Related papers  (2025-07-28T16:18:29Z)
• CAST: Cross-Attentive Spatio-Temporal feature fusion for Deepfake detection [0.0]
  CNNs are effective at capturing spatial artifacts, and Transformers excel at modeling temporal inconsistencies.<n>We propose a unified CAST model that leverages cross-attention to effectively fuse spatial and temporal features.<n>We evaluate the performance of our model using the FaceForensics++, Celeb-DF, and DeepfakeDetection datasets.
  arXiv  Detail & Related papers  (2025-06-26T18:51:17Z)
• Context-aware TFL: A Universal Context-aware Contrastive Learning Framework for Temporal Forgery Localization [60.73623588349311]
  We propose a universal context-aware contrastive learning framework (UniCaCLF) for temporal forgery localization.<n>Our approach leverages supervised contrastive learning to discover and identify forged instants by means of anomaly detection.<n>An efficient context-aware contrastive coding is introduced to further push the limit of instant feature distinguishability between genuine and forged instants.
  arXiv  Detail & Related papers  (2025-06-10T06:40:43Z)
• EHGCN: Hierarchical Euclidean-Hyperbolic Fusion via Motion-Aware GCN for Hybrid Event Stream Perception [57.54227977352417]
  Event cameras emit high-speed event stream for perception tasks.<n>We propose a novel approach named EHGCN to perceive event stream in both Euclidean and hyperbolic spaces.
  arXiv  Detail & Related papers  (2025-04-23T11:01:03Z)
• Spatiotemporal Learning with Context-aware Video Tubelets for Ultrasound Video Analysis [4.611737599608456]
  We propose a lightweight framework for tubelet-based object detection and video classifying. To address the loss of global context, we embed tubelet location, size, and confidence as inputs to the classifier. Our method is efficient, with the tubelet comprising only 0.4M parameters.
  arXiv  Detail & Related papers  (2025-03-21T18:39:42Z)
• Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance [37.61183525419993]
  3D Semantic Scene Completion (SSC) provides comprehensive scene geometry and semantics for autonomous driving perception. Existing SSC methods are limited to capturing sparse information from the current frame or naively stacking multi-frame temporal features. We propose a novel temporal SSC method FlowScene: Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance.
  arXiv  Detail & Related papers  (2025-02-20T12:52:36Z)
• Generalizable Deepfake Detection via Effective Local-Global Feature Extraction [5.221473306027505]
  GANs and diffusion models have led to the generation of increasingly realistic fake images. Deepfake detection has become a pressing issue in today's world. We propose a novel method that effectively combines local and global features.
  arXiv  Detail & Related papers  (2025-01-25T15:53:57Z)
• Leveraging Consistent Spatio-Temporal Correspondence for Robust Visual Odometry [7.517597541959445]
  We introduce S-Temporal Visual Odometry (STVO), a novel deep network architecture to enhance accuracy and consistency of multi-frame flow matching.<n>Our STVO achieves state-the-art performance on ETH3D benchmark and 38.9% on KITTI Odometry benchmark over the previous best methods.
  arXiv  Detail & Related papers  (2024-12-22T08:47:13Z)
• TSdetector: Temporal-Spatial Self-correction Collaborative Learning for Colonoscopy Video Detection [19.00902297385955]
  We propose a novel Temporal-Spatial self-correction detector (TSdetector), which integrates temporal-level consistency learning and spatial-level reliability learning to detect objects continuously. The experimental results on three publicly available polyp video dataset show that TSdetector achieves the highest polyp detection rate and outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-09-30T06:19:29Z)
• SSRFlow: Semantic-aware Fusion with Spatial Temporal Re-embedding for Real-world Scene Flow [6.995663556921384]
  Scene flow provides the 3D motion field of the first frame from two consecutive point clouds. We propose a novel approach called Dual Cross Attentive (DCA) for the latent fusion and alignment between two frames based semantic contexts. We leverage novel domain adaptive losses to effectively bridge the gap of motion inference from synthetic to real-world.
  arXiv  Detail & Related papers  (2024-07-31T02:28:40Z)
• Typhoon Intensity Prediction with Vision Transformer [51.84456610977905]
  We introduce "Typhoon Intensity Transformer" (Tint) to predict typhoon intensity accurately across space and time. Tint uses self-attention mechanisms with global receptive fields per layer. Experiments on a publicly available typhoon benchmark validate the efficacy of Tint.
  arXiv  Detail & Related papers  (2023-11-28T03:11:33Z)
• Unearthing Common Inconsistency for Generalisable Deepfake Detection [8.327980745153216]
  Video-level one shows its potential to have both generalization across multiple domains and robustness to compression. We propose a detection approach by capturing frame inconsistency that broadly exists in different forgery techniques. We introduce a temporally-preserved module method to introduce spatial noise perturbations, directing the model's attention towards temporal information.
  arXiv  Detail & Related papers  (2023-11-20T06:04:09Z)
• Local-Global Temporal Difference Learning for Satellite Video Super-Resolution [55.69322525367221]
  We propose to exploit the well-defined temporal difference for efficient and effective temporal compensation. To fully utilize the local and global temporal information within frames, we systematically modeled the short-term and long-term temporal discrepancies. Rigorous objective and subjective evaluations conducted across five mainstream video satellites demonstrate that our method performs favorably against state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-04-10T07:04:40Z)
• Blur Interpolation Transformer for Real-World Motion from Blur [52.10523711510876]
  We propose a encoded blur transformer (BiT) to unravel the underlying temporal correlation in blur. Based on multi-scale residual Swin transformer blocks, we introduce dual-end temporal supervision and temporally symmetric ensembling strategies. In addition, we design a hybrid camera system to collect the first real-world dataset of one-to-many blur-sharp video pairs.
  arXiv  Detail & Related papers  (2022-11-21T13:10:10Z)
• Delving into Sequential Patches for Deepfake Detection [64.19468088546743]
  Recent advances in face forgery techniques produce nearly untraceable deepfake videos, which could be leveraged with malicious intentions. Previous studies has identified the importance of local low-level cues and temporal information in pursuit to generalize well across deepfake methods. We propose the Local- & Temporal-aware Transformer-based Deepfake Detection framework, which adopts a local-to-global learning protocol.
  arXiv  Detail & Related papers  (2022-07-06T16:46:30Z)
• Video Salient Object Detection via Adaptive Local-Global Refinement [7.723369608197167]
  Video salient object detection (VSOD) is an important task in many vision applications. We propose an adaptive local-global refinement framework for VSOD. We show that our weighting methodology can further exploit the feature correlations, thus driving the network to learn more discriminative feature representation.
  arXiv  Detail & Related papers  (2021-04-29T14:14:11Z)
• Temporal Context Aggregation Network for Temporal Action Proposal Refinement [93.03730692520999]
  Temporal action proposal generation is a challenging yet important task in the video understanding field. Current methods still suffer from inaccurate temporal boundaries and inferior confidence used for retrieval. We propose TCANet to generate high-quality action proposals through "local and global" temporal context aggregation.
  arXiv  Detail & Related papers  (2021-03-24T12:34:49Z)
• Exploring Rich and Efficient Spatial Temporal Interactions for Real Time Video Salient Object Detection [87.32774157186412]
  Main stream methods formulate their video saliency mainly from two independent venues, i.e., the spatial and temporal branches. In this paper, we propose atemporal network to achieve such improvement in a full interactive fashion. Our method is easy to implement yet effective, achieving high quality video saliency detection in real-time speed with 50 FPS.
  arXiv  Detail & Related papers  (2020-08-07T03:24:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.