Deep Dreams Are Made of This: Visualizing Monosemantic Features in Diffusion Models

• URL: http://arxiv.org/abs/2605.08218v1
• Date: Wed, 06 May 2026 13:21:24 GMT
• Title: Deep Dreams Are Made of This: Visualizing Monosemantic Features in Diffusion Models
• Abstract summary: This paper proposes latent visualization by optimization (LVO), a mechanistic interpretability technique that extends feature visualization by optimization to latent diffusion models.<n>LVO employs sparse autoencoders (SAEs) to disentangle polysemantic layer representations into monosemantic features.
• Score: 0.12277343096128711
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper proposes latent visualization by optimization (LVO), a mechanistic interpretability technique that extends feature visualization by optimization - originally developed for convolutional neural networks - to latent diffusion models. LVO employs sparse autoencoders (SAEs) to disentangle polysemantic layer representations into monosemantic features. Key contributions include latent-space optimization, time-step activity analysis, schedule-matched noise injection, prior initialization through feature steering, and suitable regularization strategies. We demonstrate the method on Stable Diffusion 1.5 fine-tuned on the Style50 dataset, showing that SAE features produce clear visualizations of recognizable concepts - including diagonal compositions, human figures, roses, cables, and waterfall foam - that correlate with dataset examples, while the baseline without disentanglement produces less coherent results. We further show that regularization techniques from pixel-space feature visualization transfer to the latent domain, though they require different configurations for the raw-layer and SAE variants. Compared to dataset examples and steering, LVO provides complementary insights by directly revealing what activates a feature rather than its downstream effects.

Related papers

• Spatial Gram Alignment for Ultra-High-Resolution Image Synthesis [38.08544812796336]
  Modern ultra-high-resolution image synthesis relies heavily on the robust generative capacity of large-scale pre-trained Latent Diffusion Models (LDMs)<n>We propose a novel framework that explicitly leverages the representation priors of vision foundation models while preserving the native generative capacity of LDMs.
  arXiv  Detail & Related papers  (2026-05-20T06:59:12Z)
• V-Co: A Closer Look at Visual Representation Alignment via Co-Denoising [65.5867130156805]
  We present V-Co, a systematic study of visual co-denoising in a unified JiT-based framework.<n>Our study reveals four key ingredients for effective visual co-denoising.<n>V-Co outperforms the underlying pixel-space diffusion baseline and strong prior pixel-diffusion methods.
  arXiv  Detail & Related papers  (2026-03-17T17:01:54Z)
• CASL: Concept-Aligned Sparse Latents for Interpreting Diffusion Models [45.90361318326864]
  Internal activations of diffusion models encode rich semantic information, but interpreting such representations remains challenging.<n>We introduce CASL (Concept-Aligned Sparse Latents), a supervised framework that aligns sparse latent dimensions of diffusion models with semantic concepts.<n>Unlike editing methods, CASL-Steer is used solely as a causal probe to reveal how concept-aligned latents influence generated content.
  arXiv  Detail & Related papers  (2026-01-21T20:14:17Z)
• UniDiff: Parameter-Efficient Adaptation of Diffusion Models for Land Cover Classification with Multi-Modal Remotely Sensed Imagery and Sparse Annotations [3.588053519843616]
  We propose a parameter-efficient framework that adapts a single ImageNet-pretrained diffusion model to multiple sensing modalities using only target-domain data.<n>UniDiff combines FiLM-based timestep-modality conditioning, parameter-efficient adaptation of approximately 5% of parameters, and pseudo-RGB anchoring to preserve pre-trained representations and prevent catastrophic forgetting.
  arXiv  Detail & Related papers  (2025-11-29T00:57:53Z)
• Latent Diffusion Model without Variational Autoencoder [78.34722551463223]
  SVG is a novel latent diffusion model without variational autoencoders for visual generation.<n>It constructs a feature space with clear semantic discriminability by leveraging frozen DINO features.<n>It enables accelerated diffusion training, supports few-step sampling, and improves generative quality.
  arXiv  Detail & Related papers  (2025-10-17T04:17:44Z)
• "Principal Components" Enable A New Language of Images [79.45806370905775]
  We introduce a novel visual tokenization framework that embeds a provable PCA-like structure into the latent token space.<n>Our approach achieves state-of-the-art reconstruction performance and enables better interpretability to align with the human vision system.
  arXiv  Detail & Related papers  (2025-03-11T17:59:41Z)
• Exploring Representation-Aligned Latent Space for Better Generation [86.45670422239317]
  We introduce ReaLS, which integrates semantic priors to improve generation performance.<n>We show that fundamental DiT and SiT trained on ReaLS can achieve a 15% improvement in FID metric.<n>The enhanced semantic latent space enables more perceptual downstream tasks, such as segmentation and depth estimation.
  arXiv  Detail & Related papers  (2025-02-01T07:42:12Z)
• How to Use Diffusion Priors under Sparse Views? [29.738350228085928]
  Inline Prior Guided Score Matching is proposed to provide visual supervision over sparse views in 3D reconstruction.<n>We show that our method achieves state-of-the-art reconstruction quality.
  arXiv  Detail & Related papers  (2024-12-03T07:31:54Z)
• Harnessing Diffusion Models for Visual Perception with Meta Prompts [68.78938846041767]
  We propose a simple yet effective scheme to harness a diffusion model for visual perception tasks. We introduce learnable embeddings (meta prompts) to the pre-trained diffusion models to extract proper features for perception. Our approach achieves new performance records in depth estimation tasks on NYU depth V2 and KITTI, and in semantic segmentation task on CityScapes.
  arXiv  Detail & Related papers  (2023-12-22T14:40:55Z)
• SatDM: Synthesizing Realistic Satellite Image with Semantic Layout Conditioning using Diffusion Models [0.0]
  Denoising Diffusion Probabilistic Models (DDPMs) have demonstrated significant promise in synthesizing realistic images from semantic layouts. In this paper, a conditional DDPM model capable of taking a semantic map and generating high-quality, diverse, and correspondingly accurate satellite images is implemented. The effectiveness of our proposed model is validated using a meticulously labeled dataset introduced within the context of this study.
  arXiv  Detail & Related papers  (2023-09-28T19:39:13Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.