Omnimodal Dataset Distillation via High-order Proxy Alignment

• URL: http://arxiv.org/abs/2604.10666v1
• Date: Sun, 12 Apr 2026 14:47:41 GMT
• Title: Omnimodal Dataset Distillation via High-order Proxy Alignment
• Abstract summary: We propose HoPA, a unified method that captures high-order cross-modal alignments via a proxy structure.<n>Our method avoids the complexity of pairwise modality modeling and enables scalable proxy analysis across heterogeneous modalities.
• Score: 59.74134776126595
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dataset distillation compresses large-scale datasets into compact synthetic sets while preserving training performance, but existing methods are largely restricted to single-modal or bimodal settings. Extending dataset distillation to scenarios involving more than two modalities, i.e., Omnimodal Dataset Distillation, remains underexplored and challenging due to increased heterogeneity and complex cross-modal interactions. In this work, we identify the key determinant that bounds the endpoint discrepancy in the omnimodal setting, which is exacerbated with an increasing number of modalities. To this end, we propose HoPA, a unified method that captures high-order cross-modal alignments via a compact proxy, which is compatible with trajectory matching as well. By abstracting omnimodal alignment with a shared similarity structure, our method avoids the combinatorial complexity of pairwise modality modeling and enables scalable joint distillation across heterogeneous modalities. Theoretical analysis from the spectral perspective reveals the rationality of our proposed method against bimodal dataset distillation techniques. Extensive experiments on various benchmarks demonstrate that the proposed method achieves superior compression-performance trade-offs compared to existing competitors. The source code will be publicly released.

Related papers

• DAIF: A Data-Driven Intermediate Fusion Framework for Multimodal Supervised Learning via Approximate Message Passing [11.222071637855079]
  DAIF is a data adaptive intermediate fusion framework to learn fusion structure directly from data.<n>We operate under a Bayesian multimodal factor model where the prior on the latent factors determines the cross-modal dependence.<n>Our method clusters modalities based on estimated intermodal dependence, then performs clusterwise empirical Bayes estimation of the priors.
  arXiv  Detail & Related papers  (2026-08-03T18:10:58Z)
• Beyond Modality Fusion: Deep Ensembles for Multimodal Classification [0.38059763597999]
  In multimodal classification, late-fusion approaches classifyd modality-specific features extracted by unimodal neural networks.<n>We show that multimodal data can be effectively classified without any explicit modality fusion, using deep ensembles of unimodal networks.<n>We propose a synthetic multimodal framework that allows control over both the number of modalities and their predictive strength.
  arXiv  Detail & Related papers  (2026-07-06T12:59:45Z)
• Multimodal Distribution Matching for Vision-Language Dataset Distillation [50.411341509805936]
  Multimodal Distribution Matching is a geometry-aware framework for efficient and generalizable multimodal distillation.<n>MDM integrates complementary components at the data, model, and loss levels.<n>It yields compact synthetic sets that preserve multimodal semantics, substantially reduce distillation cost, and remain robust across architectures.
  arXiv  Detail & Related papers  (2026-05-22T10:41:58Z)
• AlignMamba-2: Enhancing Multimodal Fusion and Sentiment Analysis with Modality-Aware Mamba [56.52470564147458]
  We propose textbfAlignMamba-2, an effective and efficient framework for multimodal fusion and sentiment analysis.<n>Our approach introduces a dual alignment strategy that regularizes the model using both Optimal Transport distance and Maximum Mean Discrepancy.<n>More importantly, we design a Modality-Aware Mamba layer, which employs a Mixture-of-Experts architecture with modality-specific and modality-shared experts.
  arXiv  Detail & Related papers  (2026-03-19T03:47:21Z)
• IMS3: Breaking Distributional Aggregation in Diffusion-Based Dataset Distillation [41.83834685276882]
  Recently, diffusion-based dataset distillation methods have shown great promise by leveraging the strong generative capacity of diffusion models to produce diverse and structurally consistent samples.<n>Inversion-Matching (IM) introduces an inversion-guided fine-tuning process that aligns denoising trajectories with their inversion counterparts, broadening distributional coverage and enhancing diversity.<n> Selective Subgroup Sampling(S3) is a training-free sampling mechanism that improves inter-class separability by selecting synthetic subsets that are both representative and distinctive.
  arXiv  Detail & Related papers  (2026-03-14T14:15:03Z)
• Complementarity-Supervised Spectral-Band Routing for Multimodal Emotion Recognition [60.20529806857076]
  Multimodal emotion recognition fuses cues such as text, video, and audio to understand individual emotional states.<n>Prior methods face two main limitations: mechanically relying on independent unimodal performance, and coarse-grained fusion conflicting with the fine-grained representations required by emotion tasks.<n>We propose the Complementarity-Supervised Multi-Band Expert Network, named Atsuko, to model fine-grained complementary features via multi-scale band decomposition and expert collaboration.
  arXiv  Detail & Related papers  (2026-03-07T03:58:48Z)
• Decoupled Audio-Visual Dataset Distillation [44.63243875072762]
  We propose DAVDD, a pretraining-based decoupled audio-visual distillation framework.<n>To address these challenges, we propose DAVDD, a pretraining-based decoupled audio-visual distillation framework.
  arXiv  Detail & Related papers  (2025-11-22T02:36:50Z)
• Hyperbolic Dataset Distillation [44.63243875072762]
  We propose a novel hyperbolic dataset distillation method.<n>We find that pruning in hyperbolic space requires only 20% of the distilled core set to retain model performance.<n>This is the first work to incorporate the hyperbolic space into the dataset distillation process.
  arXiv  Detail & Related papers  (2025-05-30T14:14:00Z)
• Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalities [16.69453837626083]
  We propose a Correlation-decoupled Knowledge Distillation (CorrKD) framework for the Multimodal Sentiment Analysis (MSA) task under uncertain missing modalities. We present a sample-level contrastive distillation mechanism that transfers comprehensive knowledge containing cross-sample correlations to reconstruct missing semantics. We design a response-disentangled consistency distillation strategy to optimize the sentiment decision boundaries of the student network.
  arXiv  Detail & Related papers  (2024-04-25T09:35:09Z)
• BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistency [66.8685113725007]
  BiCro aims to estimate soft labels for noisy data pairs to reflect their true correspondence degree. experiments on three popular cross-modal matching datasets demonstrate that BiCro significantly improves the noise-robustness of various matching models.
  arXiv  Detail & Related papers  (2023-03-22T09:33:50Z)
• Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis [96.46952672172021]
  Bi-Bimodal Fusion Network (BBFN) is a novel end-to-end network that performs fusion on pairwise modality representations. Model takes two bimodal pairs as input due to known information imbalance among modalities.
  arXiv  Detail & Related papers  (2021-07-28T23:33:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.