Decoupled Audio-Visual Dataset Distillation

• URL: http://arxiv.org/abs/2511.17890v1
• Date: Sat, 22 Nov 2025 02:36:50 GMT
• Title: Decoupled Audio-Visual Dataset Distillation
• Authors: Wenyuan Li, Guang Li, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama,
• Abstract summary: We propose DAVDD, a pretraining-based decoupled audio-visual distillation framework.<n>To address these challenges, we propose DAVDD, a pretraining-based decoupled audio-visual distillation framework.
• Score: 44.63243875072762
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Audio-Visual Dataset Distillation aims to compress large-scale datasets into compact subsets while preserving the performance of the original data. However, conventional Distribution Matching (DM) methods struggle to capture intrinsic cross-modal alignment. Subsequent studies have attempted to introduce cross-modal matching, but two major challenges remain: (i) independently and randomly initialized encoders lead to inconsistent modality mapping spaces, increasing training difficulty; and (ii) direct interactions between modalities tend to damage modality-specific (private) information, thereby degrading the quality of the distilled data. To address these challenges, we propose DAVDD, a pretraining-based decoupled audio-visual distillation framework. DAVDD leverages a diverse pretrained bank to obtain stable modality features and uses a lightweight decoupler bank to disentangle them into common and private representations. To effectively preserve cross-modal structure, we further introduce Common Intermodal Matching together with a Sample-Distribution Joint Alignment strategy, ensuring that shared representations are aligned both at the sample level and the global distribution level. Meanwhile, private representations are entirely isolated from cross-modal interaction, safeguarding modality-specific cues throughout distillation. Extensive experiments across multiple benchmarks show that DAVDD achieves state-of-the-art results under all IPC settings, demonstrating the effectiveness of decoupled representation learning for high-quality audio-visual dataset distillation. Code will be released.

Related papers

• Test-time Adaptive Hierarchical Co-enhanced Denoising Network for Reliable Multimodal Classification [55.56234913868664]
  We propose Test-time Adaptive Hierarchical Co-enhanced Denoising Network (TAHCD) for reliable learning on multimodal data.<n>The proposed method achieves superior classification performance, robustness, and generalization compared with state-of-the-art reliable multimodal learning approaches.
  arXiv  Detail & Related papers  (2026-01-12T03:14:12Z)
• Modality-Specific Enhancement and Complementary Fusion for Semi-Supervised Multi-Modal Brain Tumor Segmentation [6.302779966909783]
  We propose a novel semi-supervised multi-modal framework for medical image segmentation.<n>We introduce a Modality-specific Enhancing Module (MEM) to strengthen semantic unique cues to each modality.<n>We also introduce a learnable Complementary Information Fusion (CIF) module to adaptively exchange complementary knowledge between modalities.
  arXiv  Detail & Related papers  (2025-12-10T16:15:17Z)
• ImagebindDC: Compressing Multi-modal Data with Imagebind-based Condensation [12.924585390383085]
  ImageBindDC is a novel data condensation framework operating within the unified feature space of ImageBind.<n>Our approach moves beyond conventional distribution-matching by employing a powerful Characteristic Function (CF) loss.<n>Experiments highlight the effectiveness of ImageBindDC: on the NYU-v2 dataset.
  arXiv  Detail & Related papers  (2025-11-11T13:55:46Z)
• Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model [62.889356203346985]
  We propose DUal-STream diffusion (DUST), a world-model augmented VLA framework that handles the modality conflict.<n>DUST achieves up to 6% gains over a standard VLA baseline and implicit world-modeling methods.<n>On real-world tasks with the Franka Research 3, DUST outperforms baselines in success rate by 13%.
  arXiv  Detail & Related papers  (2025-10-31T16:32:12Z)
• Towards General Modality Translation with Contrastive and Predictive Latent Diffusion Bridge [16.958159611661813]
  Latent Denoising Diffusion Bridge Model (LDDBM) is a general-purpose framework for modality translation.<n>By operating in a shared latent space, our method learns a bridge between arbitrary modalities without requiring aligned dimensions.<n>Our approach supports arbitrary modality pairs and performs strongly on diverse MT tasks, including multi-view to 3D shape generation, image super-resolution, and multi-view scene synthesis.
  arXiv  Detail & Related papers  (2025-10-23T17:59:54Z)
• Dual-granularity Sinkhorn Distillation for Enhanced Learning from Long-tailed Noisy Data [67.25796812343454]
  Real-world datasets for deep learning frequently suffer from the co-occurring challenges of class imbalance and label noise.<n>We propose Dual-granularity Sinkhorn Distillation (D-SINK), a novel framework that enhances dual robustness by distilling and integrating complementary insights.<n>Experiments on benchmark datasets demonstrate that D-SINK significantly improves robustness and achieves strong empirical performance in learning from long-tailed noisy data.
  arXiv  Detail & Related papers  (2025-10-09T13:05:27Z)
• High-Quality Sound Separation Across Diverse Categories via Visually-Guided Generative Modeling [65.02357548201188]
  We propose DAVIS, a Diffusion-based Audio-VIsual Separation framework that solves the audio-visual sound source separation task through generative learning.<n>Our framework operates by synthesizing the desired separated sound spectrograms directly from a noise distribution, conditioned concurrently on the mixed audio input and associated visual information.
  arXiv  Detail & Related papers  (2025-09-26T08:46:00Z)
• Multimodal Variational Auto-encoder based Audio-Visual Segmentation [46.67599800471001]
  ECMVAE factorizes the representations of each modality with a modality-shared representation and a modality-specific representation. Our approach leads to a new state-of-the-art for audio-visual segmentation, with a 3.84 mIOU performance leap.
  arXiv  Detail & Related papers  (2023-10-12T13:09:40Z)
• Ensemble Modeling for Multimodal Visual Action Recognition [50.38638300332429]
  We propose an ensemble modeling approach for multimodal action recognition. We independently train individual modality models using a variant of focal loss tailored to handle the long-tailed distribution of the MECCANO [21] dataset.
  arXiv  Detail & Related papers  (2023-08-10T08:43:20Z)
• Complete Cross-triplet Loss in Label Space for Audio-visual Cross-modal Retrieval [7.459223771397159]
  Cross-modal data (e.g. audiovisual) have different distributions and representations that cannot be directly compared. To bridge the gap between audiovisual modalities, we learn a common subspace for them by utilizing the intrinsic correlation in the natural synchronization of audio-visual data with the aid of annotated labels. We propose a new AV-CMR model to optimize semantic features by directly predicting labels and then measuring the intrinsic correlation between audio-visual data using complete cross-triple loss.
  arXiv  Detail & Related papers  (2022-11-07T10:37:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.