VRAG-DFD: Verifiable Retrieval-Augmentation for MLLM-based Deepfake Detection

• URL: http://arxiv.org/abs/2604.13660v3
• Date: Fri, 17 Apr 2026 10:24:38 GMT
• Title: VRAG-DFD: Verifiable Retrieval-Augmentation for MLLM-based Deepfake Detection
• Abstract summary: We propose the VRAG-DFD framework with accurate dynamic forgery knowledge retrieval and powerful critical reasoning capabilities.<n>In terms of data, we constructed two datasets with RAG: Forensic Knowledge Database (FKD) for DFD knowledge annotation, and Forensic Chain-of-Thought dataset (F-CoT) for critical CoT construction.<n>In terms of performance, VRAG-DFD achieved SOTA and competitive performance on DFD generalization testing.
• Score: 49.88962095967614
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In Deepfake Detection (DFD) tasks, researchers proposed two types of MLLM-based methods: complementary combination with small DFD detectors, or static forgery knowledge injection. The lack of professional forgery knowledge hinders the performance of these DFD-MLLMs. To solve this, we deeply considered two insightful issues: How to provide high-quality associated forgery knowledge for MLLMs? AND How to endow MLLMs with critical reasoning abilities given noisy reference information? Notably, we attempted to address above two questions with preliminary answers by leveraging the combination of Retrieval-Augmented Generation (RAG) and Reinforcement Learning (RL). Through RAG and RL techniques, we propose the VRAG-DFD framework with accurate dynamic forgery knowledge retrieval and powerful critical reasoning capabilities. Specifically, in terms of data, we constructed two datasets with RAG: Forensic Knowledge Database (FKD) for DFD knowledge annotation, and Forensic Chain-of-Thought Dataset (F-CoT), for critical CoT construction. In terms of model training, we adopt a three-stage training method (Alignment->SFT->GRPO) to gradually cultivate the critical reasoning ability of the MLLM. In terms of performance, VRAG-DFD achieved SOTA and competitive performance on DFD generalization testing.

Related papers

• Foundation and Multimodal Large Language Models for Face Presentation and Morph Attack Detection [36.05932340652857]
  Presentation attack detection (PAD) and morphing attack detection (MAD) are essential components of trustworthy face biometrics.<n>We investigate whether general-purpose foundation models (FMs) and multimodal large language models (MLLMs) encode PAD-relevant and MAD-relevant information.<n>Experiments show that FMs and MLLMs can achieve significant performance for PAD and MAD.
  arXiv  Detail & Related papers  (2026-08-30T14:07:31Z)
• RADAR: Revealing Asymmetric Development of Abilities in MLLM Pre-training [59.493415006017635]
  Pre-trained Multi-modal Large Language Models (MLLMs) provide a knowledge-rich foundation for post-training.<n>Current evaluation relies on testing after supervised fine-tuning, which introduces laborious additional training and autoregressive decoding costs.<n>We propose RADAR, an efficient ability-centric evaluation framework for Revealing Asymmetric Development of Abilities in MLLM pRe-training.
  arXiv  Detail & Related papers  (2026-02-13T12:56:31Z)
• Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection [59.04089915447622]
  ForenAgent is an interactive IFD framework that enables MLLMs to autonomously generate, execute, and refine Python-based low-level tools around the detection objective.<n>Inspired by human reasoning, we design a dynamic reasoning loop comprising global perception, local focusing, iterative probing, and holistic adjudication.<n>Experiments show that ForenAgent exhibits emergent tool-use competence and reflective reasoning on challenging IFD tasks.
  arXiv  Detail & Related papers  (2025-12-18T08:38:44Z)
• Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thought [58.321044666612174]
  Vad-R1 is an end-to-end MLLM-based framework for Video Anomaly Reasoning.<n>We design a Perception-to-Cognition Chain-of-Thought (P2C-CoT) that simulates the human process of recognizing anomalies.<n>We also propose an improved reinforcement learning algorithm AVA-GRPO, which explicitly incentivizes the anomaly reasoning capability of MLLMs.
  arXiv  Detail & Related papers  (2025-05-26T12:05:16Z)
• DMRL: Data- and Model-aware Reward Learning for Data Extraction [3.511535517476954]
  Large language models (LLMs) are inherently vulnerable to unintended privacy breaches.<n>We propose a Data- and Model-aware Reward Learning approach for data extraction.
  arXiv  Detail & Related papers  (2025-05-07T07:21:37Z)
• Unlocking the Capabilities of Large Vision-Language Models for Generalizable and Explainable Deepfake Detection [18.125287697902813]
  Current Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities in understanding multimodal data.<n>We present a novel framework that unlocks LVLMs' potential capabilities for deepfake detection.
  arXiv  Detail & Related papers  (2025-03-19T03:20:03Z)
• R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning [87.30285670315334]
  textbfR1-Searcher is a novel two-stage outcome-based RL approach designed to enhance the search capabilities of Large Language Models.<n>Our framework relies exclusively on RL, without requiring process rewards or distillation for a cold start.<n>Our experiments demonstrate that our method significantly outperforms previous strong RAG methods, even when compared to the closed-source GPT-4o-mini.
  arXiv  Detail & Related papers  (2025-03-07T17:14:44Z)
• Towards Interactive Deepfake Analysis [40.0271474912034]
  This paper aims to explore interactive deepfake analysis by performing instruction tuning on multi-modal large language models (MLLMs)<n>To address these issues, we introduce (1) a GPT-assisted data construction process resulting in an instruction-following dataset called DFA-Instruct, (2) a benchmark named DFA-Bench, designed to comprehensively evaluate the capabilities of MLLMs in deepfake detection, deepfake classification, and artifact description, and (3) construct an interactive deepfake analysis system called DFA-GPT, as a strong baseline for the community.
  arXiv  Detail & Related papers  (2025-01-02T09:34:11Z)
• FD-LLM: Large Language Model for Fault Diagnosis of Machines [20.679299204776527]
  This study introduces a novel IFD approach by effectively adapting large language models to numerical data inputs for identifying faults from time-series sensor data.<n>We propose FD-LLM, an LLM framework specifically designed for fault diagnosis by formulating the training of the LLM as a multi-class classification problem.<n>We assess the fault diagnosis capabilities of four open-sourced LLMs based on the FD-LLM framework, and evaluate the models' adaptability and generalizability under various operational conditions.
  arXiv  Detail & Related papers  (2024-12-02T07:36:35Z)
• X2-DFD: A framework for eXplainable and eXtendable Deepfake Detection [55.77552681618732]
  X2-DFD is an eXplainable and eXtendable framework based on multimodal large-language models (MLLMs) for deepfake detection.<n>The first stage, Model Feature Assessment, systematically evaluates the detectability of forgery-related features for the MLLM.<n>The second stage, Explainable dataset Construction, consists of two key modules: Strong Feature Strengthening and Weak Feature Supplementing.<n>The third stage, Fine-tuning and Inference, involves fine-tuning the MLLM on the constructed dataset and deploying it for final detection and explanation.
  arXiv  Detail & Related papers  (2024-10-08T15:28:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.