Multimodal Distribution Matching for Vision-Language Dataset Distillation

• URL: http://arxiv.org/abs/2605.23482v1
• Date: Fri, 22 May 2026 10:41:58 GMT
• Title: Multimodal Distribution Matching for Vision-Language Dataset Distillation
• Abstract summary: Multimodal Distribution Matching is a geometry-aware framework for efficient and generalizable multimodal distillation.<n>MDM integrates complementary components at the data, model, and loss levels.<n>It yields compact synthetic sets that preserve multimodal semantics, substantially reduce distillation cost, and remain robust across architectures.
• Score: 50.411341509805936
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Dataset distillation compresses large training sets into compact synthetic datasets while preserving downstream performance. As modern systems increasingly operate on paired vision-language inputs, multimodal distillation must preserve representation quality and cross-modal alignment under tight compute and memory budgets, yet prior methods often require heavy computes and overlook their correlations. To address this, we present Multimodal Distribution Matching (MDM), a geometry-aware framework for efficient and generalizable multimodal distillation. Specifically, MDM integrates complementary components at the data, model, and loss levels. At the data level, it initializes synthetic image-text pairs by sampling from clusters in the joint embedding space. At the model level, it forms a mixed teacher by interpolating independently fine-tuned models in weight space according to their angular deviation from the pretrained anchor. At the loss level, it matches joint distributions on the unit hypersphere using a geometry-aware matching objective that exploits the joint features in the cross-modal agreement and discrepancy directions along with symmetric contrastive learning. Across image-text retrieval benchmarks with cross-architecture evaluation, MDM yields compact synthetic sets that preserve multimodal semantics, substantially reduce distillation cost, and remain robust across architectures.

Related papers

• Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation [54.74045834035952]
  Vision-language dataset distillation (VLDD) compresses a large image-text paired dataset into a small set of synthetic pairs.<n>We propose a rank-aware hyperbolic alignment (RAHA) that combines hierarchical geometry with explicit alignment-capacity control.<n>RAHA demonstrates competitive cross-modal retrieval and improved transfer indicators under fixed budgets.
  arXiv  Detail & Related papers  (2026-06-28T15:41:31Z)
• Geometry-Aware Dataset Condensation for Diffusion Model Training [103.45641113998839]
  We propose to reformulate real subset selection as a geometry-aware distribution alignment problem.<n>By incorporating one-sided partial optimal transport, our method selectively aligns a compact subset with the full data distribution.<n>Experiments across diffusion variants, subset sizes, image resolutions, and training rounds show that our method achieves superior fidelity and distributional coverage.
  arXiv  Detail & Related papers  (2026-06-04T08:53:58Z)
• Omnimodal Dataset Distillation via High-order Proxy Alignment [59.74134776126595]
  We propose HoPA, a unified method that captures high-order cross-modal alignments via a proxy structure.<n>Our method avoids the complexity of pairwise modality modeling and enables scalable proxy analysis across heterogeneous modalities.
  arXiv  Detail & Related papers  (2026-04-12T14:47:41Z)
• AlignMamba-2: Enhancing Multimodal Fusion and Sentiment Analysis with Modality-Aware Mamba [56.52470564147458]
  We propose textbfAlignMamba-2, an effective and efficient framework for multimodal fusion and sentiment analysis.<n>Our approach introduces a dual alignment strategy that regularizes the model using both Optimal Transport distance and Maximum Mean Discrepancy.<n>More importantly, we design a Modality-Aware Mamba layer, which employs a Mixture-of-Experts architecture with modality-specific and modality-shared experts.
  arXiv  Detail & Related papers  (2026-03-19T03:47:21Z)
• Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillation [109.13471554184554]
  We reformulate dataset distillation as an Optimal Transport (OT) distance minimization problem.<n>OT offers a geometrically faithful framework for distribution matching.<n>Our method consistently outperforms state-of-the-art approaches in an efficient manner.
  arXiv  Detail & Related papers  (2025-11-29T04:04:05Z)
• ImagebindDC: Compressing Multi-modal Data with Imagebind-based Condensation [12.924585390383085]
  ImageBindDC is a novel data condensation framework operating within the unified feature space of ImageBind.<n>Our approach moves beyond conventional distribution-matching by employing a powerful Characteristic Function (CF) loss.<n>Experiments highlight the effectiveness of ImageBindDC: on the NYU-v2 dataset.
  arXiv  Detail & Related papers  (2025-11-11T13:55:46Z)
• Hyperbolic Dataset Distillation [44.63243875072762]
  We propose a novel hyperbolic dataset distillation method.<n>We find that pruning in hyperbolic space requires only 20% of the distilled core set to retain model performance.<n>This is the first work to incorporate the hyperbolic space into the dataset distillation process.
  arXiv  Detail & Related papers  (2025-05-30T14:14:00Z)
• Dataset Distillation with Probabilistic Latent Features [9.318549327568695]
  A compact set of synthetic data can effectively replace the original dataset in downstream classification tasks.<n>We propose a novel approach that models the joint distribution of latent features.<n>Our method achieves state-of-the-art cross architecture performance across a range of backbone architectures.
  arXiv  Detail & Related papers  (2025-05-10T13:53:49Z)
• VRS-UIE: Value-Driven Reordering Scanning for Underwater Image Enhancement [104.78586859995333]
  State Space Models (SSMs) have emerged as a promising backbone for vision tasks due to their linear complexity and global receptive field.<n>The predominance of large-portion, homogeneous but useless oceanic backgrounds can dilute the feature representation responses of sparse yet valuable targets.<n>We propose a novel Value-Driven Reordering Scanning framework for Underwater Image Enhancement (UIE)<n>Our framework sets a new state-of-the-art, delivering superior enhancement performance (surpassing WMamba by 0.89 dB on average) by effectively suppressing water bias and preserving structural and color fidelity.
  arXiv  Detail & Related papers  (2025-05-02T12:21:44Z)
• MDM: Advancing Multi-Domain Distribution Matching for Automatic Modulation Recognition Dataset Synthesis [35.07663680944459]
  Deep learning technology has been successfully introduced into Automatic Modulation Recognition (AMR) tasks. The success of deep learning is all attributed to the training on large-scale datasets. In order to solve the problem of large amount of data, some researchers put forward the method of data distillation.
  arXiv  Detail & Related papers  (2024-08-05T14:16:54Z)
• D$^4$M: Dataset Distillation via Disentangled Diffusion Model [4.568710926635445]
  We propose an efficient framework for dataset distillation via Disentangled Diffusion Model (D$4$M) Compared to architecture-dependent methods, D$4$M employs latent diffusion model to guarantee consistency and incorporates label information into category prototypes. D$4$M demonstrates superior performance and robust generalization, surpassing the SOTA methods across most aspects.
  arXiv  Detail & Related papers  (2024-07-21T12:16:20Z)
• Contrastive Learning-Based Spectral Knowledge Distillation for Multi-Modality and Missing Modality Scenarios in Semantic Segmentation [2.491548070992611]
  novel multi-modal fusion approach called CSK-Net is proposed. It uses a contrastive learning-based spectral knowledge distillation technique. Experiments show that CSK-Net surpasses state-of-the-art models in multi-modal tasks and for missing modalities.
  arXiv  Detail & Related papers  (2023-12-04T10:27:09Z)
• RGM: A Robust Generalizable Matching Model [49.60975442871967]
  We propose a deep model for sparse and dense matching, termed RGM (Robust Generalist Matching) To narrow the gap between synthetic training samples and real-world scenarios, we build a new, large-scale dataset with sparse correspondence ground truth. We are able to mix up various dense and sparse matching datasets, significantly improving the training diversity.
  arXiv  Detail & Related papers  (2023-10-18T07:30:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.