Latent Transfer Attack: Adversarial Examples via Generative Latent Spaces

• URL: http://arxiv.org/abs/2603.06311v1
• Date: Fri, 06 Mar 2026 14:14:28 GMT
• Title: Latent Transfer Attack: Adversarial Examples via Generative Latent Spaces
• Abstract summary: Adrial attacks are a central tool for probing the robustness of modern vision models.<n>Most methods optimize perturbations directly in pixel space under $ell_infty$ or $ell$ constraints.<n>We propose $textbfLTA$ ($textbfL$atent $textbfT$ransfer $textbfA$ttack), a transfer-based attack that instead optimize perturbations in the latent space of a pretrained Stable Diffusion VAE.
• Score: 36.919974069544146
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adversarial attacks are a central tool for probing the robustness of modern vision models, yet most methods optimize perturbations directly in pixel space under $\ell_\infty$ or $\ell_2$ constraints. While effective in white-box settings, pixel-space optimization often produces high-frequency, texture-like noise that is brittle to common preprocessing (e.g., resizing and cropping) and transfers poorly across architectures. We propose $\textbf{LTA}$ ($\textbf{L}$atent $\textbf{T}$ransfer $\textbf{A}$ttack), a transfer-based attack that instead optimizes perturbations in the latent space of a pretrained Stable Diffusion VAE. Given a clean image, we encode it into a latent code and optimize the latent representation to maximize a surrogate classifier loss, while softly enforcing a pixel-space $\ell_\infty$ budget after decoding. To improve robustness to resolution mismatch and standard input pipelines, we incorporate Expectation Over Transformations (EOT) via randomized resizing, interpolation, and cropping, and apply periodic latent Gaussian smoothing to suppress emerging artifacts and stabilize optimization. Across a suite of CNN and vision-transformer targets, LTA achieves strong transfer attack success while producing spatially coherent, predominantly low-frequency perturbations that differ qualitatively from pixel-space baselines and occupy a distinct point in the transfer-quality trade-off. Our results highlight pretrained generative latent spaces as an effective and structured domain for adversarial optimization, bridging robustness evaluation with modern generative priors.

Related papers

• Isotropic Embedding Perturbations for Robust Vision Language Encoders [73.92626712781859]
  Aether is a simple plug-in method that applies diffusion-style random perturbations in the embedding space via controlled alpha-mixing.<n>Inspired by feature-space perturbations in language models and image degradation in generative pretraining, Aether induces mild yet effective perturbations.<n>Aether delivers consistent gains over the advanced recipe combining CutMix, Mixup, DropPath, and RandAug.
  arXiv  Detail & Related papers  (2026-09-09T15:08:05Z)
• Hybrid-Domain Posterior Sampling for Inverse Problems via Latent Flow Matching [22.663919850054928]
  Hybrid-Domain Posterior Sampling (HDPS) is a decoupled inference framework that disentangles physical measurement consistency from semantic prior modeling.<n> Experiments on diverse inverse problems demonstrate that HDPS establishes a new state-of-the-art, successfully recovering the high-frequency structural precision that latent-only solvers inherently discard.
  arXiv  Detail & Related papers  (2026-08-01T08:46:12Z)
• Group-of-Latents: Perceptual Video Compression at Extreme Bitrates via Masked Latent Generative Modeling [49.11379121512358]
  We propose a unified generative framework that leverages pre-trained Diffusion Transformer priors to achieve high perceptual quality at extremely lows.<n>Our method achieves state-of-the-art perceptual fidelity with rich spatial details and robust temporal consistency.
  arXiv  Detail & Related papers  (2026-07-21T06:36:32Z)
• Evolving Layer-Specific Scalar Functions for Hardware-Aware Transformer Adaptation [0.5266471639657487]
  Vision Transformers (ViTs) achieve state-of-the-art performance on challenging vision tasks, but their deployment on edge devices is hindered.<n>Recent methods attempt to bypass this by replacing normalization layers with hardware-friendly scalar approximations.<n>We propose a highly efficient framework that utilizes genetic programming (GP) to evolve heterogeneous, layer-specific scalar functions.
  arXiv  Detail & Related papers  (2026-05-13T19:08:55Z)
• Oracle Noise: Faster Semantic Spherical Alignment for Interpretable Latent Optimization [5.814544128372275]
  Oracle Noise is a zero-shot framework reframing noise as semantic-driven optimization strictly confined to a hypersphere.<n>It significantly accelerates semantic alignment and superior aesthetics without black-box models.<n>It achieves state-of-the-art performance across human preference metrics (e.g., HPSv2, ImageReward), semantic alignment (CLIP Score), and sample diversity, all within a strict 2-second optimization budget.
  arXiv  Detail & Related papers  (2026-04-26T05:32:26Z)
• Multi-Scale Local Speculative Decoding for Image Generation [10.239314110594249]
  We introduce Multi-Scale Local Speculative Decoding (MuLo-SD)<n>MuLo-SD combines multi-resolution drafting with spatially informed verification to accelerate AR image generation.<n>We demonstrate that MuLo-SD achieves substantial speedups up to $mathbf1.7times$.
  arXiv  Detail & Related papers  (2026-01-08T17:39:35Z)
• Generative Latent Coding for Ultra-Low Bitrate Image Compression [61.71793017252801]
  We introduce a Generative Latent Coding architecture, which performs transform coding in the latent space of a generative vector-quantized variational auto-encoder (VQ-VAE), instead of in the pixel space.<n>The generative latent space is characterized by greater sparsity, richer semantic and better alignment with human perception, rendering it advantageous for achieving high-realism and high-fidelity compression.
  arXiv  Detail & Related papers  (2025-12-23T09:35:40Z)
• Denoising Vision Transformer Autoencoder with Spectral Self-Regularization [21.85836384863372]
  We show that redundant high-frequency components in high-dimensional latent spaces hinder the training convergence of diffusion models.<n>We propose a spectral self-regularization strategy to suppress redundant high-frequency noise while simultaneously preserving reconstruction quality.<n>The resulting Denoising-VAE, a ViT-based autoencoder, produces cleaner, lower-noise latents, leading to improved generative quality and faster optimization convergence.
  arXiv  Detail & Related papers  (2025-11-16T15:00:32Z)
• ExGS: Extreme 3D Gaussian Compression with Diffusion Priors [60.7245825868903]
  We introduce ExGS and GaussPainter for Extreme 3DGS compression.<n>GassPainter fills in missing regions and enhances visible pixels, yielding substantial improvements in degraded renderings.<n>Our framework can even achieve over 100X compression (reducing a typical 354.77 MB model to about 3.31 MB)
  arXiv  Detail & Related papers  (2025-09-29T13:23:06Z)
• GreedyPixel: Fine-Grained Black-Box Adversarial Attack Via Greedy Algorithm [21.84393608348216]
  GreedyPixel is a new adversarial attack framework for deep neural networks.<n>It combines a surrogate-derived pixel priority map with greedy, per-pixel optimization refined by query feedback.<n>Our results show that GreedyPixel bridges the precision gap between white-box and black-box attacks.
  arXiv  Detail & Related papers  (2025-01-24T04:17:03Z)
• Sampling-based Fast Gradient Rescaling Method for Highly Transferable Adversarial Attacks [19.917677500613788]
  gradient-based approaches generally use the $sign$ function to generate perturbations at the end of the process. We propose a Sampling-based Fast Gradient Rescaling Method (S-FGRM) to improve the transferability of crafted adversarial examples.
  arXiv  Detail & Related papers  (2022-04-06T15:12:20Z)
• Transferable Sparse Adversarial Attack [62.134905824604104]
  We introduce a generator architecture to alleviate the overfitting issue and thus efficiently craft transferable sparse adversarial examples. Our method achieves superior inference speed, 700$times$ faster than other optimization-based methods.
  arXiv  Detail & Related papers  (2021-05-31T06:44:58Z)
• Patch-wise++ Perturbation for Adversarial Targeted Attacks [132.58673733817838]
  We propose a patch-wise iterative method (PIM) aimed at crafting adversarial examples with high transferability. Specifically, we introduce an amplification factor to the step size in each iteration, and one pixel's overall gradient overflowing the $epsilon$-constraint is properly assigned to its surrounding regions. Compared with the current state-of-the-art attack methods, we significantly improve the success rate by 35.9% for defense models and 32.7% for normally trained models.
  arXiv  Detail & Related papers  (2020-12-31T08:40:42Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.