DiffSafeMerge: Mitigating Backdoor Inheritance in Diffusion Model Merging

• URL: http://arxiv.org/abs/2608.09445v1
• Date: Mon, 10 Aug 2026 11:20:35 GMT
• Title: DiffSafeMerge: Mitigating Backdoor Inheritance in Diffusion Model Merging
• Abstract summary: Mitigation is difficult without knowing the compromised source, trigger, or target.<n>We introduce DiffSafeMerge (DSM), which uses a small unlabeled clean set and fixed, attack-agnostic stress probes to score source blocks.
• Score: 12.412925762132284
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unconditional diffusion checkpoint merging assumes benign sources, yet a compromised public checkpoint can transfer a dormant backdoor while clean generation appears normal. Mitigation is difficult without knowing the compromised source, trigger, or target, and broad sanitization may degrade image quality. We introduce DiffSafeMerge (DSM), which uses a small unlabeled clean set and fixed, attack-agnostic stress probes to score source blocks, shrink suspicious contributions toward a trusted reference, and select attenuation under a clean denoising-loss budget. We evaluate four attacks, two datasets, and 21 target conditions. Intended merging already has zero worst-target ASR in 10 of 14 source cases; DSM preserves these outcomes and records no target match in the remaining four over three seeds, including three with baseline ASR of 48--100\%. Among methods with zero worst-target ASR on both datasets, DSM obtains the lowest case-averaged FID in the matched seed-0 comparison.

Related papers

• Subgroup Membership Inference Audits of Differentially Private Synthetic Text [28.947903737268916]
  Even when the worst-case privacy leakage of such releases is bounded by means of differential privacy (DP), in practice a residual risk remains.<n>This paper defines a subgroup-targeted membership inference game in which the target pool is an explicit parameter, and instantiates it with an audit of 32 proxies.<n>The audit shows that synthetic releases leak subgroup membership and that prior attacks systematically underestimate this leakage.
  arXiv  Detail & Related papers  (2026-09-09T07:56:24Z)
• MaliciousSkillBench: A Comprehensive Benchmark for Malicious Agent Skill Detection [24.37473229751569]
  MaliciousSkillBench is a comprehensive benchmark for malicious Agent Skill detection.<n>We consolidate 13 public sources, 11 of which contribute Core malicious artifacts, and reduce 8,414 raw malicious records to 7,539 normalized-unique identities.<n>We then evaluate three learned text detectors and three off-the-shelf Skill scanners.
  arXiv  Detail & Related papers  (2026-08-20T11:13:00Z)
• Inference-Time Consensus for Mitigating Hidden Behaviors from LLM Fine-Tuning [6.382763634670965]
  We introduce two consensus decoders: a token-wise minimum, which caps each token at the lowest probability any source assigns, and a base-relative variant, which reverts to the base probability on any token the sources move in opposing directions.<n>Across controlled poisoning tasks, subliminal learning, and emergent misalignment, consensus decoding suppresses source-specific misbehavior while preserving shared desirable behavior.
  arXiv  Detail & Related papers  (2026-07-25T23:41:47Z)
• A Failure-Mode Benchmark for Polymorphic Sybil Poisoning in RAG [4.879573049082894]
  We release a benchmark and failure-mode-aware evaluation framework for grounded QA under coordinated retrieval poisoning.<n>The framework partitions reader outputs into four mutually exclusive categories (emphgold, emphhijack, emphabstention, emphdrift)<n>We introduce emphpolymorphic sybil poisoning, a coordinated attack class in which $S$ lexically diverse passages jointly support an attacker-chosen target.
  arXiv  Detail & Related papers  (2026-07-04T06:56:06Z)
• Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglement [66.67463557274358]
  Foundation models trained on biased datasets often rely on spurious correlations between target labels and non-causal attributes.<n>We introduce Confidence-guided Concept Mining (CBCM) to identify reliable spurious attributes without attribute annotations.<n>We propose Dual-branch Cross-projection Debiasing (DCD), a prompt-tuning framework that separates target and spurious representations into two branches.
  arXiv  Detail & Related papers  (2026-06-23T05:28:47Z)
• What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility [48.10132234701036]
  We propose a diagnostic evaluation protocol based on three checks -- scale stability, reference validity, and depth utility -- together with per-layer rather than aggregate cosine reporting.<n>Across multiple architectures and methods, our protocol identifies all failures with wide calibration margins.
  arXiv  Detail & Related papers  (2026-06-19T06:04:17Z)
• Towards Persistent Case-Based Memory for Autonomous Data Science: A CBR-Augmented R&D-Agent with a Locally Deployable Small Language Model [0.0]
  We present CBR-augmented R&D-Agent, integrating a persistent CBR layer into Microsoft's R&D-Agent framework with a custom backend for Gemma 4 31B Dense.<n>Cases are stored as structured records with executable code snapshots and quality metadata.<n>Heuristic reuse detection across 108 retrieval events shows high semantic relevance.
  arXiv  Detail & Related papers  (2026-06-03T12:56:11Z)
• SilentRetrieval: Hijacking Retrieval-Augmented Generation via Semantically-Preserving Adversarial Data Poisoning [1.0998907972211756]
  We present SilentRetrieval, a two-stage data poisoning attack that hijacks RAG systems through adversarially crafted documents.<n>SilentRetrieval achieves 84.6%/81.3% HR@10 and 57.5%/54.8% ASR-LLM on Natural Questions and MS MARCO, while maintaining near-benign perplexity.
  arXiv  Detail & Related papers  (2026-05-27T07:30:30Z)
• Toward Efficient Membership Inference Attacks against Federated Large Language Models: A Projection Residual Approach [44.738736515079346]
  Federated Large Language Models (FedLLMs) enable multiple parties to collaboratively fine-tune LLMs without sharing raw data.<n>FedLLMs' unique properties, i.e. massive parameter scales, rapid convergence, and sparse, non-orthogonal gradients, render existing MIAs ineffective.<n>We propose ProjRes, the first projection residuals-based passive MIA tailored for FedLLMs.
  arXiv  Detail & Related papers  (2026-04-23T01:44:04Z)
• VeriX-Anon: A Multi-Layered Framework for Mathematically Verifiable Outsourced Target-Driven Data Anonymization [0.0]
  VeriX-Anon is a multi-layered verification framework for outsourced Target-Driven k-anonymization.<n>It combines deterministic verification via Merkle-style hashing of an Authenticated Decision Tree, probabilistic verification via Boundary Sentinels near the Random Forest decision boundary and exact-duplicate Twins with cryptographic identifiers.<n>It correctly detected deviations in 11 of 12 scenarios.
  arXiv  Detail & Related papers  (2026-04-14T08:22:18Z)
• From Black Box to Glass Box: Cross-Model ASR Disagreement to Prioto Review in Ambient AI Scribe Documentation [43.148402136307716]
  Cross-model disagreement among heterogeneous ASR systems can act as a reference-free uncertainty signal.<n>We transcribed 50 publicly available medical audio clips with eight ASR systems spanning commercial APIs and open-source engines.<n>Low-agreement regions were enriched for content disagreements, with the content fraction increasing from 53.9% to 73.9% across quintiles of high-risk mass.
  arXiv  Detail & Related papers  (2026-03-02T13:02:13Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• ME: Trigger Element Combination Backdoor Attack on Copyright Infringement [76.0062084678398]
  SilentBadDiffusion (SBD) is a method proposed recently, which shew outstanding performance in attacking SD in text-to-image tasks.<n>In this paper, we raised new datasets accessible for researching in attacks like SBD, and proposed Multi-Element (ME) attack method based on SBD.<n>The Copyright Infringement Rate (CIR) / First Attack Epoch (FAE) we got on the two new datasets were 16.78% / 39.50 and 51.20% / 23.60, respectively close to or even outperformed benchmark Pokemon and Mijourney datasets.
  arXiv  Detail & Related papers  (2025-06-12T14:51:27Z)
• Discriminator-Free Generative Adversarial Attack [87.71852388383242]
  Agenerative-based adversarial attacks can get rid of this limitation. ASymmetric Saliency-based Auto-Encoder (SSAE) generates the perturbations. The adversarial examples generated by SSAE not only make thewidely-used models collapse, but also achieves good visual quality.
  arXiv  Detail & Related papers  (2021-07-20T01:55:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.