CAST: Collapse-Aware multi-Scale Topology Fusion for Multimodal Coreset Selection

• URL: http://arxiv.org/abs/2605.11705v1
• Date: Tue, 12 May 2026 07:59:08 GMT
• Title: CAST: Collapse-Aware multi-Scale Topology Fusion for Multimodal Coreset Selection
• Abstract summary: We propose a Collapse-Aware multi-Scale Topology fusion framework for multimodal coreset selection.<n>We first construct image- and text-modality topologies, and derive a unified topology via local-collapse-aware refinement and cross-modal fusion.<n>We then introduce a multi-scale distribution matching criterion in the diffusion wavelet domain, encouraging the coreset to approximate the original dataset at multiple scales.
• Score: 8.275673045109079
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The training of large multimodal models fundamentally relies on massive image-text datasets, which inevitably incur prohibitive computational overhead. Dataset selection offers a promising paradigm by identifying a highly informative coreset. However, existing approaches suffer from two critical limitations: (i) single-modality-dominated sampling methods, which ignore the fine-grained cross-modal information imbalance inherent in multimodal datasets and thus lead to semantic loss in the other modality; and (ii) coarse-grained sample-scoring-based sampling methods, where the selected coreset tends to be biased toward the scoring model, making it difficult to guarantee distributional equivalence between the coreset and the original dataset. Meanwhile, existing distribution matching and discrete sampling strategies often fail to jointly account for global semantic structure, local fine-grained details, and redundancy-aware coverage in dense regions. To this end, we propose CAST, a Collapse-Aware multi-Scale Topology fusion framework for multimodal coreset selection. We first construct image- and text-modality topologies, and derive a unified topology via local-collapse-aware refinement and cross-modal fusion. We then introduce a multi-scale distribution matching criterion in the diffusion wavelet domain, encouraging the coreset to approximate the original dataset at multiple scales. Finally, we introduce a local soft relational coverage mechanism that extends pure geometric coverage to relation-aware indirect coverage, penalizing redundant selections in dense clusters. Extensive experiments on Flickr30K and MS-COCO show that CAST outperforms existing dataset selection baselines, showcasing great superiority in cross-architecture generalization and energy efficiency over state-of-the-art multimodal synthesis methods.

Related papers

• Deep Multimodal Fusion Detection through Spatial Mask and Channel Fusion [4.909774323398836]
  This paper proposes an Attention-Driven Complementarity Resampling framework for robust improvement of cross-modality object detection.<n>Based on a shared channel spatial attention mechanism, we first introduce the semantic mask exchange to actively mix the boundaries of the modalities.<n>Then we propose a learnable channel competition to sample and aggregate features in a channel-wise and learnable way.
  arXiv  Detail & Related papers  (2026-08-03T11:51:12Z)
• MUNI: Multimodal Unified Latent Diffusion for Coherent Any-to-Any Generation [26.862922663735347]
  MUNI is an end-to-end multimodal latent diffusion framework for any-to-any generation.<n>It unifies subset-conditioned cross-modal generation and unconditional joint sampling through a shared latent.<n>MUNI rests on two complementary contributions, one architectural and one in the training objective.
  arXiv  Detail & Related papers  (2026-06-15T08:46:48Z)
• Geometry-Aware Dataset Condensation for Diffusion Model Training [103.45641113998839]
  We propose to reformulate real subset selection as a geometry-aware distribution alignment problem.<n>By incorporating one-sided partial optimal transport, our method selectively aligns a compact subset with the full data distribution.<n>Experiments across diffusion variants, subset sizes, image resolutions, and training rounds show that our method achieves superior fidelity and distributional coverage.
  arXiv  Detail & Related papers  (2026-06-04T08:53:58Z)
• Disentangled Fine-Grained Prototype Learning for Incomplete Image-Tabular Classification [28.25868366778138]
  DFPL is a novel framework for fine-grained prototype learning.<n>SSPM extracts compact and diverse shared and modality-specific prototypes.<n>PFA module jointly enforces prototype-level distribution matching and prototype-to-class semantic alignment.
  arXiv  Detail & Related papers  (2026-06-03T21:24:05Z)
• PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging [80.17966517536102]
  We argue that the core of Multimodal Large Language Models pre-training lies in establishing effective cross-modal alignment.<n>Motivated by this insight, we introduce the post-alignment merging task, which aims to integrate cross-modal alignment capabilities learned from multimodal pre-training.<n>We propose PivotMerge, a post-alignment merging framework for cross-modal projectors.
  arXiv  Detail & Related papers  (2026-04-18T09:38:03Z)
• Towards One-for-All Anomaly Detection for Tabular Data [87.63505963517512]
  Tabular anomaly detection (TAD) is critical in many real-world applications.<n>We propose OFA-TAD, a framework that only requires one-time training on multiple source datasets.<n>Experiments on 34 datasets from 14 domains demonstrate that OFA-TAD achieves superior anomaly detection performance.
  arXiv  Detail & Related papers  (2026-03-15T14:42:05Z)
• UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting [90.47915032778366]
  We propose UniDiff, a unified diffusion framework for multimodal time series forecasting.<n>At its core lies a unified and parallel fusion module, where a single cross-attention mechanism integrates structural information from timestamps and semantic context from texts.<n>Experiments on real-world benchmark datasets across eight domains demonstrate that the proposed UniDiff model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-08T05:36:14Z)
• Localized Kernel Projection Outlyingness: A Two-Stage Approach for Multi-Modal Outlier Detection [0.0]
  Two-Stage LKPLO is a novel multi-stage outlier detection framework.<n>It overcomes the coexisting limitations of conventional projection-based methods.<n>It achieves state-of-the-art performance on challenging datasets.
  arXiv  Detail & Related papers  (2025-10-28T03:53:46Z)
• FindRec: Stein-Guided Entropic Flow for Multi-Modal Sequential Recommendation [57.577843653775]
  We propose textbfFindRec (textbfFlexible unified textbfinformation textbfdisentanglement for multi-modal sequential textbfRecommendation)<n>A Stein kernel-based Integrated Information Coordination Module (IICM) theoretically guarantees distribution consistency between multimodal features and ID streams.<n>A cross-modal expert routing mechanism that adaptively filters and combines multimodal features based on their contextual relevance.
  arXiv  Detail & Related papers  (2025-07-07T04:09:45Z)
• Multisource Collaborative Domain Generalization for Cross-Scene Remote Sensing Image Classification [57.945437355714155]
  Cross-scene image classification aims to transfer prior knowledge of ground materials to annotate regions with different distributions.<n>Existing approaches focus on single-source domain generalization to unseen target domains.<n>We propose a novel multi-source collaborative domain generalization framework (MS-CDG) based on homogeneity and heterogeneity characteristics of multi-source remote sensing data.
  arXiv  Detail & Related papers  (2024-12-05T06:15:08Z)
• Few-shot Multimodal Sentiment Analysis based on Multimodal Probabilistic Fusion Prompts [30.15646658460899]
  Multimodal sentiment analysis has gained significant attention due to the proliferation of multimodal content on social media. Existing studies in this area rely heavily on large-scale supervised data, which is time-consuming and labor-intensive to collect. We propose a novel method called Multimodal Probabilistic Fusion Prompts (MultiPoint) that leverages diverse cues from different modalities for multimodal sentiment detection in the few-shot scenario.
  arXiv  Detail & Related papers  (2022-11-12T08:10:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.