Real-Aware Residual Model Merging for Deepfake Detection

• URL: http://arxiv.org/abs/2509.24367v1
• Date: Mon, 29 Sep 2025 07:10:03 GMT
• Title: Real-Aware Residual Model Merging for Deepfake Detection
• Abstract summary: Deepfake generators evolve quickly, making exhaustive data collection and repeated retraining impractical.<n>We propose Real-aware Residual Model Merging (R$2$M), a training-free parameter-space merging framework.<n>R$2$M estimates a shared Real component via a low-rank factorization of task vectors, decomposes each specialist into a Real-aligned part and a Fake residual, denoises residuals with layerwise rank truncation, and aggregates them with per-task norm matching.
• Score: 29.454462278073063
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deepfake generators evolve quickly, making exhaustive data collection and repeated retraining impractical. We argue that model merging is a natural fit for deepfake detection: unlike generic multi-task settings with disjoint labels, deepfake specialists share the same binary decision and differ in generator-specific artifacts. Empirically, we show that simple weight averaging preserves Real representations while attenuating Fake-specific cues. Building upon these findings, we propose Real-aware Residual Model Merging (R$^2$M), a training-free parameter-space merging framework. R$^2$M estimates a shared Real component via a low-rank factorization of task vectors, decomposes each specialist into a Real-aligned part and a Fake residual, denoises residuals with layerwise rank truncation, and aggregates them with per-task norm matching to prevent any single generator from dominating. A concise rationale explains why a simple head suffices: the Real component induces a common separation direction in feature space, while truncated residuals contribute only minor off-axis variations. Across in-distribution, cross-dataset, and unseen-dataset, R$^2$M outperforms joint training and other merging baselines. Importantly, R$^2$M is also composable: when a new forgery family appears, we fine-tune one specialist and re-merge, eliminating the need for retraining.

Related papers

• FairReL: Deepfake Detection using Fairness-Aware Representation Learning [13.693865200913741]
  We propose a fairness-aware representation-learning framework that targets subgroup-sensitive components.<n>FairReL uses an SVD-decomposed foundation-model backbone to isolate the fine-tuning-induced residual representation.<n>Experiments on FF++, Celeb-DF, DFD and DFDC show that, against the state-of-the-art fairness-aware detector, FairReL improves unseen-dataset AUC by 3.9% while reducing subgroup FPR disparity by 10.2%.
  arXiv  Detail & Related papers  (2026-08-28T18:34:15Z)
• UniRank: Unified Rank Allocation for Low-Rank LLM Compression [9.742379971053806]
  We formulate global low-rank allocation as a sorting-and-truncation pipeline.<n>We verify the strong correlation between high input-output cosine similarity and low effective rank.<n>We propose rank-preserving fine-tuning, which performs direct LoRA tuning on decomposed weights.
  arXiv  Detail & Related papers  (2026-06-20T03:10:33Z)
• Closed-Form Spectral Regularization for Multi-Task Model Merging [96.82449201305234]
  Model merging combines several independently fine-tuned experts into a single multi-task model without any training data.<n>State-of-the-art merging methods formulate merging as a layer-wise interference problem.<n>We propose SWUDI, a closed-form method that combines a soft exponential filter, which matches the gradient-flow trajectory of iterative descent.
  arXiv  Detail & Related papers  (2026-06-05T14:00:47Z)
• ResMerge: Residual-based Spectral Merging of Large Language Models [52.65011413018957]
  We propose ResMerge, a residual-based spectral merging framework for RL experts.<n>Based on this observation, we propose ResMerge, a residual-based spectral merging framework for RL experts.
  arXiv  Detail & Related papers  (2026-06-01T13:42:45Z)
• Amortized Variational Inference for Logistic Regression with Missing Covariates [4.027766797152636]
  Amortized Variational Inference for Logistic Regression (AV-LR) is a unified end-to-end framework for binary logistic regression.<n>AV-LR integrates a probabilistic generative model with a simple amortized inference network, trained jointly by maximizing the evidence lower bound.<n>It achieves estimation accuracy comparable to or better than state-of-the-art EM-like algorithms, with significantly lower computational cost.
  arXiv  Detail & Related papers  (2026-03-22T14:00:59Z)
• One-Shot Federated Ridge Regression: Exact Recovery via Sufficient Statistic Aggregation [0.7106986689736825]
  Federated ridge regression is a distributed equilibrium problem where each client computes local sufficient statistics and transmits them once.<n>We establish differential privacy guarantees where noise is injected once per client, eliminating the composition penalty that degrades privacy in multi-round protocols.<n>Experiments on synthetic heterogeneous regression demonstrate that one-shot fusion matches FedAvg accuracy while requiring up to $38times$ less communication.
  arXiv  Detail & Related papers  (2026-01-13T04:47:22Z)
• Mixture of Ranks with Degradation-Aware Routing for One-Step Real-World Image Super-Resolution [76.66229730098759]
  In real-world image super-resolution (Real-ISR), existing approaches mainly rely on fine-tuning pre-trained diffusion models.<n>We propose a Mixture-of-Ranks (MoR) architecture for single-step image super-resolution.<n>We introduce a fine-grained expert partitioning strategy that treats each rank in LoRA as an independent expert.
  arXiv  Detail & Related papers  (2025-11-20T04:11:44Z)
• ShortcutBreaker: Low-Rank Noisy Bottleneck with Global Perturbation Attention for Multi-Class Unsupervised Anomaly Detection [59.89803740308262]
  ShortcutBreaker is a novel unified feature-reconstruction framework for MUAD tasks.<n>It features two key innovations to address the issue of shortcuts.<n>The proposed method achieves a remarkable image-level AUROC of 99.8%, 98.9%, 90.6%, and 87.8% on four datasets.
  arXiv  Detail & Related papers  (2025-10-21T06:51:30Z)
• ResAD++: Towards Class Agnostic Anomaly Detection via Residual Feature Learning [52.11294707895649]
  This paper explores the problem of class-agnostic anomaly detection (AD)<n>The objective is to train one class-agnostic AD model that can generalize to detect anomalies in diverse new classes from different domains without any retraining or fine-tuning on the target data.<n> Comprehensive experiments on eight real-world AD datasets demonstrate that our ResAD++ can achieve remarkable AD results when directly used in new classes.
  arXiv  Detail & Related papers  (2025-09-28T08:41:05Z)
• DDL: A Large-Scale Datasets for Deepfake Detection and Localization in Diversified Real-World Scenarios [51.916287988122406]
  We present a novel large-scale deepfake detection and localization (textbfDDL) dataset containing over $textbf1.4M+$ forged samples.<n>Our DDL not only provides a more challenging benchmark for complex real-world forgeries but also offers crucial support for building next-generation deepfake detection, localization, and interpretability methods.
  arXiv  Detail & Related papers  (2025-06-29T15:29:03Z)
• Reinforced Model Merging [53.84354455400038]
  We present an innovative framework termed Reinforced Model Merging (RMM), which encompasses an environment and agent tailored for merging tasks.<n>By utilizing data subsets during the evaluation process, we addressed the bottleneck in the reward feedback phase, thereby accelerating RMM by up to 100 times.
  arXiv  Detail & Related papers  (2025-03-27T08:52:41Z)
• Sample-Efficient Linear Representation Learning from Non-IID Non-Isotropic Data [4.971690889257356]
  We introduce an adaptation of the alternating minimization-descent scheme proposed by Collins and Nayer and Vaswani. We show that vanilla alternating-minimization descent fails catastrophically even for iid, but mildly non-isotropic data. Our analysis unifies and generalizes prior work, and provides a flexible framework for a wider range of applications.
  arXiv  Detail & Related papers  (2023-08-08T17:56:20Z)
• Connective Reconstruction-based Novelty Detection [3.7706789983985303]
  Deep learning has enabled us to analyze real-world data which contain unexplained samples. GAN-based approaches have been widely used to address this problem due to their ability to perform distribution fitting. We propose a simple yet efficient reconstruction-based method that avoids adding complexities to compensate for the limitations of GAN models.
  arXiv  Detail & Related papers  (2022-10-25T11:09:39Z)
• Recovery Guarantees for Distributed-OMP [8.393317912360564]
  We study distributed schemes for high-dimensional sparse linear regression. We prove that distributed-OMP schemes recover the support of the regression vector with communication per machine linear in its sparsity and logarithmic in the dimension. Our simulations show that distributed-OMP schemes are competitive with more computationally intensive methods, and in some cases even outperform them.
  arXiv  Detail & Related papers  (2022-09-15T11:43:33Z)
• Federated Latent Class Regression for Hierarchical Data [5.110894308882439]
  Federated Learning (FL) allows a number of agents to participate in training a global machine learning model without disclosing locally stored data. We propose a novel probabilistic model, Hierarchical Latent Class Regression (HLCR), and its extension to Federated Learning, FEDHLCR. Our inference algorithm, being derived from Bayesian theory, provides strong convergence guarantees and good robustness to overfitting. Experimental results show that FEDHLCR offers fast convergence even in non-IID datasets.
  arXiv  Detail & Related papers  (2022-06-22T00:33:04Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• Distributionally Robust Multi-Output Regression Ranking [3.9318191265352196]
  We introduce a new listwise listwise learning-to-rank model called Distributionally Robust Multi-output Regression Ranking (DRMRR) DRMRR uses a Distributionally Robust Optimization framework to minimize a multi-output loss function under the most adverse distributions in the neighborhood of the empirical data distribution. Our experiments were conducted on two real-world applications, medical document retrieval, and drug response prediction.
  arXiv  Detail & Related papers  (2021-09-27T05:19:27Z)
• Examining and Combating Spurious Features under Distribution Shift [94.31956965507085]
  We define and analyze robust and spurious representations using the information-theoretic concept of minimal sufficient statistics. We prove that even when there is only bias of the input distribution, models can still pick up spurious features from their training data. Inspired by our analysis, we demonstrate that group DRO can fail when groups do not directly account for various spurious correlations.
  arXiv  Detail & Related papers  (2021-06-14T05:39:09Z)
• Overcoming Catastrophic Forgetting with Gaussian Mixture Replay [79.0660895390689]
  We present a rehearsal-based approach for continual learning (CL) based on Gaussian Mixture Models (GMM) We mitigate catastrophic forgetting (CF) by generating samples from previous tasks and merging them with current training data. We evaluate GMR on multiple image datasets, which are divided into class-disjoint sub-tasks.
  arXiv  Detail & Related papers  (2021-04-19T11:41:34Z)
• MMCGAN: Generative Adversarial Network with Explicit Manifold Prior [78.58159882218378]
  We propose to employ explicit manifold learning as prior to alleviate mode collapse and stabilize training of GAN. Our experiments on both the toy data and real datasets show the effectiveness of MMCGAN in alleviating mode collapse, stabilizing training, and improving the quality of generated samples.
  arXiv  Detail & Related papers  (2020-06-18T07:38:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.