Spatial-Temporal Frequency Forgery Clue for Video Forgery Detection in
VIS and NIR Scenario
- URL: http://arxiv.org/abs/2207.01906v1
- Date: Tue, 5 Jul 2022 09:27:53 GMT
- Title: Spatial-Temporal Frequency Forgery Clue for Video Forgery Detection in
VIS and NIR Scenario
- Authors: Yukai Wang, Chunlei Peng, Decheng Liu, Nannan Wang and Xinbo Gao
- Abstract summary: Existing face forgery detection methods based on frequency domain find that the GAN forged images have obvious grid-like visual artifacts in the frequency spectrum compared to the real images.
This paper proposes a Cosine Transform-based Forgery Clue Augmentation Network (FCAN-DCT) to achieve a more comprehensive spatial-temporal feature representation.
- Score: 87.72258480670627
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In recent years, with the rapid development of face editing and generation,
more and more fake videos are circulating on social media, which has caused
extreme public concerns. Existing face forgery detection methods based on
frequency domain find that the GAN forged images have obvious grid-like visual
artifacts in the frequency spectrum compared to the real images. But for
synthesized videos, these methods only confine to single frame and pay little
attention to the most discriminative part and temporal frequency clue among
different frames. To take full advantage of the rich information in video
sequences, this paper performs video forgery detection on both spatial and
temporal frequency domains and proposes a Discrete Cosine Transform-based
Forgery Clue Augmentation Network (FCAN-DCT) to achieve a more comprehensive
spatial-temporal feature representation. FCAN-DCT consists of a backbone
network and two branches: Compact Feature Extraction (CFE) module and Frequency
Temporal Attention (FTA) module. We conduct thorough experimental assessments
on two visible light (VIS) based datasets WildDeepfake and Celeb-DF (v2), and
our self-built video forgery dataset DeepfakeNIR, which is the first video
forgery dataset on near-infrared modality. The experimental results demonstrate
the effectiveness of our method on detecting forgery videos in both VIS and NIR
scenarios.
Related papers
- Detecting Inpainted Video with Frequency Domain Insights [0.0]
We propose the Frequency Domain Insights Network (FDIN), which significantly enhances detection accuracy.
Previous evaluations on public datasets demonstrate that FDIN achieves state-of-the-art performance.
arXiv Detail & Related papers (2024-09-21T01:51:07Z) - Frequency Domain Nuances Mining for Visible-Infrared Person
Re-identification [75.87443138635432]
Existing methods mainly exploit the spatial information while ignoring the discriminative frequency information.
We propose a novel Frequency Domain Nuances Mining (FDNM) method to explore the cross-modality frequency domain information.
Our method outperforms the second-best method by 5.2% in Rank-1 accuracy and 5.8% in mAP on the SYSU-MM01 dataset.
arXiv Detail & Related papers (2024-01-04T09:19:54Z) - No-frills Temporal Video Grounding: Multi-Scale Neighboring Attention
and Zoom-in Boundary Detection [52.03562682785128]
Temporal video grounding aims to retrieve the time interval of a language query from an untrimmed video.
A significant challenge in TVG is the low "Semantic Noise Ratio (SNR)", which results in worse performance with lower SNR.
We propose a no-frills TVG model that consists of two core modules, namely multi-scale neighboring attention and zoom-in boundary detection.
arXiv Detail & Related papers (2023-07-20T04:12:10Z) - Adaptive Frequency Learning in Two-branch Face Forgery Detection [66.91715092251258]
We propose Adaptively learn Frequency information in the two-branch Detection framework, dubbed AFD.
We liberate our network from the fixed frequency transforms, and achieve better performance with our data- and task-dependent transform layers.
arXiv Detail & Related papers (2022-03-27T14:25:52Z) - TBNet:Two-Stream Boundary-aware Network for Generic Image Manipulation
Localization [49.521622399483846]
We propose a novel end-to-end two-stream boundary-aware network (abbreviated as TBNet) for generic image manipulation localization.
The proposed TBNet can significantly outperform state-of-the-art generic image manipulation localization methods in terms of both MCC and F1.
arXiv Detail & Related papers (2021-08-10T08:22:05Z) - Frame-rate Up-conversion Detection Based on Convolutional Neural Network
for Learning Spatiotemporal Features [7.895528973776606]
This paper proposes a frame-rate conversion detection network (FCDNet) that learns forensic features caused by FRUC in an end-to-end fashion.
FCDNet uses a stack of consecutive frames as the input and effectively learns artifacts using network blocks to learn features.
arXiv Detail & Related papers (2021-03-25T08:47:46Z) - Deep Video Inpainting Detection [95.36819088529622]
Video inpainting detection localizes an inpainted region in a video both spatially and temporally.
VIDNet, Video Inpainting Detection Network, contains a two-stream encoder-decoder architecture with attention module.
arXiv Detail & Related papers (2021-01-26T20:53:49Z) - Fake Visual Content Detection Using Two-Stream Convolutional Neural
Networks [14.781702606707642]
We propose a two-stream convolutional neural network architecture called TwoStreamNet to complement frequency and spatial domain features.
The proposed detector has demonstrated significant performance improvement compared to the current state-of-the-art fake content detectors.
arXiv Detail & Related papers (2021-01-03T18:05:07Z) - Two-branch Recurrent Network for Isolating Deepfakes in Videos [17.59209853264258]
We present a method for deepfake detection based on a two-branch network structure.
One branch propagates the original information, while the other branch suppresses the face content.
Our two novel components show promising results on the FaceForensics++, Celeb-DF, and Facebook's DFDC preview benchmarks.
arXiv Detail & Related papers (2020-08-08T01:38:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.