EVLF: Early Vision-Language Fusion for Generative Dataset Distillation

• URL: http://arxiv.org/abs/2603.07476v1
• Date: Sun, 08 Mar 2026 05:34:07 GMT
• Title: EVLF: Early Vision-Language Fusion for Generative Dataset Distillation
• Abstract summary: We introduce an Early Vision-Language Fusion (EVLF) method that aligns textual and visual embeddings at the transition between the encoder and the generative backbone.<n>EVLF is plug-and-play and can be easily integrated into any diffusion-based dataset distillation pipeline with an encoder.<n>Experiments demonstrate that EVLF generates semantically faithful and visually coherent synthetic data, yielding consistent improvements in downstream classification accuracy across varied settings.
• Score: 6.913041518668593
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dataset distillation (DD) aims to synthesize compact training sets that enable models to achieve high accuracy with significantly fewer samples. Recent diffusion-based DD methods commonly introduce semantic guidance through late-stage cross-attention, where textual prompts tend to dominate the generative process. Although this strategy enforces label relevance, it diminishes the contribution of visual latents, resulting in over-corrected samples that mirror prompt patterns rather than reflecting intrinsic visual features. To solve this problem, we introduce an Early Vision-Language Fusion (EVLF) method that aligns textual and visual embeddings at the transition between the encoder and the generative backbone. By incorporating a lightweight cross-attention module at this transition, the early representations simultaneously encode local textures and global semantic directions across the denoising process. Importantly, EVLF is plug-and-play and can be easily integrated into any diffusion-based dataset distillation pipeline with an encoder. It works across different denoiser architectures and sampling schedules without any task-specific modifications. Extensive experiments demonstrate that EVLF generates semantically faithful and visually coherent synthetic data, yielding consistent improvements in downstream classification accuracy across varied settings. Source code is available at https://github.com/wenqi-cai297/earlyfusion-for-dd/.

Related papers

• MLLM-Guided Semantic Correction for Text-to-Video Generation [66.7723669725808]
  We introduce a training-free, interpretable mid-generation correction framework that integrates multimodal large language model feedback.<n>We propose two key modules: a Semantic Assessment Supervisor that generates intermediate preview frames for semantic evaluations and deviation diagnostics, and a Semantic Modification Assistant that corrects semantic drift during inference.<n>Our method improves semantic alignment, visual fidelity, and temporal consistency without modifying model parameters.
  arXiv  Detail & Related papers  (2026-08-17T12:54:10Z)
• Anchoring and Steering Diffusion: Enhancing the Faithfulness of Text-to-Image Generation at Inference Time [16.04233421396159]
  AnchorSteer is a training-free framework that exerts fine-grained control over.<n>textbfAnchorSteer and its components.<n>Experiments on GenEval and T2I-CompBench++ demonstrate that AnchorSteer consistently outperforms existing baselines in text--image alignment.
  arXiv  Detail & Related papers  (2026-07-29T09:08:57Z)
• TextLDM: Language Modeling with Continuous Latent Diffusion [89.69255520673248]
  Diffusion Transformers (DiT) trained with flow matching in a VAE latent space have unified visual generation across images and videos.<n>We propose TextLDM, which transfers the visual latent diffusion recipe to text generation with minimal architectural modification.
  arXiv  Detail & Related papers  (2026-05-08T13:54:34Z)
• Deep Dreams Are Made of This: Visualizing Monosemantic Features in Diffusion Models [0.12277343096128711]
  This paper proposes latent visualization by optimization (LVO), a mechanistic interpretability technique that extends feature visualization by optimization to latent diffusion models.<n>LVO employs sparse autoencoders (SAEs) to disentangle polysemantic layer representations into monosemantic features.
  arXiv  Detail & Related papers  (2026-05-06T13:21:24Z)
• Visual Disentangled Diffusion Autoencoders: Scalable Counterfactual Generation for Foundation Models [1.3535770763481902]
  Foundation models, despite their robust zero-shot capabilities, remain vulnerable to spurious correlations and 'Clever Hans' strategies.<n>We propose Visual Disentangled Diffusion Autoencoders (DiDAE), a novel framework integrating frozen foundation models with disentangled dictionary learning.<n>DiDAE first edits foundation model embeddings in interpretable disentangled directions of the disentangled dictionary and then decodes them via a diffusion autoencoder.
  arXiv  Detail & Related papers  (2026-01-29T15:25:37Z)
• Diffusion-DRF: Differentiable Reward Flow for Video Diffusion Fine-Tuning [72.16213872139748]
  Diffusion-DRF is a differentiable reward flow for fine-tuning video diffusion models.<n>It backpropagates VLM feedback through the diffusion denoising chain.<n>It improves video quality and semantic alignment while mitigating reward hacking and collapse.
  arXiv  Detail & Related papers  (2026-01-07T18:05:08Z)
• Context-Aware Initialization for Reducing Generative Path Length in Diffusion Language Models [0.0]
  Large Language Models (DLLMs) enable fully parallel token decoding but often remain impractical at inference time.<n>Most existing acceleration methods focus on traversing this generative trajectory more efficiently via improved solvers or sampling strategies.<n>We propose a training-free interface that injects prompt-conditioned priors from a lightweight auxiliary model into the diffusion initialization.<n>Because injected priors can be imperfect and unmask-only decoding can over-commit early, we also introduce a simple confidence-based remasking mechanism as a form of prior skepticism.
  arXiv  Detail & Related papers  (2025-12-22T03:45:04Z)
• Semantics Lead the Way: Harmonizing Semantic and Texture Modeling with Asynchronous Latent Diffusion [59.30874672131483]
  Latent Diffusion Models (LDMs) inherently follow a coarse-to-fine generation process.<n>Recent advances have integrated semantic priors from pretrained visual encoders to further enhance LDMs.<n>We propose Semantic-First Diffusion (SFD), a latent diffusion paradigm that explicitly prioritizes semantic formation.
  arXiv  Detail & Related papers  (2025-12-04T15:57:27Z)
• EDITS: Enhancing Dataset Distillation with Implicit Textual Semantics [12.818622596576775]
  EDITS is a novel framework that exploits the implicit textual semantics within the image data to achieve enhanced distillation.<n>In this paper, we propose EDITS, a novel framework that exploits the implicit textual semantics within the image data to achieve enhanced distillation.
  arXiv  Detail & Related papers  (2025-09-17T09:48:39Z)
• Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies [62.653984010274485]
  Vision-Language-Action (VLA) models adapt large vision-language backbones to map images and instructions into robot actions.<n> prevailingAs either generate actions auto-regressively in a fixed left-to-right order or attach separate or diffusion heads outside the backbone.<n>We present Discrete Diffusion VLA, a unified-transformer policy that models discretized action chunks with discrete diffusion.
  arXiv  Detail & Related papers  (2025-08-27T17:39:11Z)
• LATTE: Latent Trajectory Embedding for Diffusion-Generated Image Detection [13.576997219135992]
  LATent Trajectory Embedding is a novel approach that models the evolution of latent embeddings across multiple denoising steps.<n> Experiments on several benchmarks, such as GenImage, Chameleon, and Diffusion Forensics, show that LATTE achieves superior performance.
  arXiv  Detail & Related papers  (2025-07-03T12:53:47Z)
• CONCORD: Concept-Informed Diffusion for Dataset Distillation [29.092857460373278]
  We propose Concept-Informed Diffusion (CONCORD) for dataset distillation.<n>The proposed method significantly enhances both the controllability and interpretability of the distilled image generation.<n>We demonstrate the efficacy of CONCORD by achieving state-of-the-art performance on ImageNet-1K and its subsets.
  arXiv  Detail & Related papers  (2025-05-23T20:39:23Z)
• Denoising Diffusion Autoencoders are Unified Self-supervised Learners [58.194184241363175]
  This paper shows that the networks in diffusion models, namely denoising diffusion autoencoders (DDAE), are unified self-supervised learners. DDAE has already learned strongly linear-separable representations within its intermediate layers without auxiliary encoders. Our diffusion-based approach achieves 95.9% and 50.0% linear evaluation accuracies on CIFAR-10 and Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-03-17T04:20:47Z)
• Unleashing Text-to-Image Diffusion Models for Visual Perception [84.41514649568094]
  VPD (Visual Perception with a pre-trained diffusion model) is a new framework that exploits the semantic information of a pre-trained text-to-image diffusion model in visual perception tasks. We show that VPD can be faster adapted to downstream visual perception tasks using the proposed VPD.
  arXiv  Detail & Related papers  (2023-03-03T18:59:47Z)
• Diffusion-Based Representation Learning [65.55681678004038]
  We augment the denoising score matching framework to enable representation learning without any supervised signal. In contrast, the introduced diffusion-based representation learning relies on a new formulation of the denoising score matching objective. Using the same approach, we propose to learn an infinite-dimensional latent code that achieves improvements of state-of-the-art models on semi-supervised image classification.
  arXiv  Detail & Related papers  (2021-05-29T09:26:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.