Hub-Spectral Activation of Latent Multimodal Knowledge

• URL: http://arxiv.org/abs/2609.17094v1
• Date: Tue, 15 Sep 2026 12:27:42 GMT
• Title: Hub-Spectral Activation of Latent Multimodal Knowledge
• Abstract summary: Hub-Spectral Activation (HSA) is a closed-form method for recovering and activating the hub-readable component of latent multimodal knowledge in frozen representations.<n>HSA composes and standardizes hub-edge statistics, extracts paired spectral directions, and combines reliability-weighted matching evidence with source-gated candidate resolution.
• Score: 48.72626098917089
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal representation learning seeks shared representations for cross-modal retrieval and knowledge transfer. Hub-based binding reduces pairwise supervision costs, but separate hub connections cannot guarantee reliable alignment between modalities without direct joint training. We introduce Hub-Spectral Activation (HSA), a closed-form method for recovering and activating the hub-readable component of latent multimodal knowledge in frozen representations. We formalize this knowledge as source-induced cross-modal dependence and characterize the component determined by the second-order statistics of two trained hub edges. Under a second-order source model, we establish conditions for exact recovery of the complete source-induced relation and bound the dimension of its hub-readable component by the hub covariance rank. HSA composes and standardizes hub-edge statistics, extracts paired spectral directions, and combines reliability-weighted matching evidence with source-gated candidate resolution for bidirectional retrieval and prototype classification. HSA requires no target-pair supervision, gradient optimization, or backbone updates. Across 19 retrieval and 11 prototype-classification relations on ImageBind and LanguageBind, HSA raises mean bidirectional Recall@10 from 18.27% to 31.15% and mean macro Top-1 accuracy from 29.01% to 52.43%, respectively. Controlled analyses further identify valid hub-edge correspondence and leading spectral directions as key sources of retrieval gains, demonstrating the utility of latent multimodal knowledge beyond native similarity scores. Code and models are publicly available at https://github.com/Luo1Yan/HSA.

Related papers

• S$^3$AM: A Single-Stream SAM with Reliability-Calibrated Frequency Adapter for Multi-modal Salient Object Detection [74.15663758681849]
  Single-stream framework integrates reliability-calibrated frequency adaptation into the adopted SAM backbone for MSOD.<n>Experiments on RGB-D, RGB-T, and RGB-NIR salient object detection benchmarks validate that the proposed framework achieves competitive performance.
  arXiv  Detail & Related papers  (2026-08-18T08:05:00Z)
• Deep Multimodal Fusion Detection through Spatial Mask and Channel Fusion [4.909774323398836]
  This paper proposes an Attention-Driven Complementarity Resampling framework for robust improvement of cross-modality object detection.<n>Based on a shared channel spatial attention mechanism, we first introduce the semantic mask exchange to actively mix the boundaries of the modalities.<n>Then we propose a learnable channel competition to sample and aggregate features in a channel-wise and learnable way.
  arXiv  Detail & Related papers  (2026-08-03T11:51:12Z)
• REAL-OW: Rehearsal-free Open World Object Detection with Low-Rank Adaptation and Dual-Stage Objectness Modeling [56.888597798431704]
  Open-World Object Detection (OWOD) requires detectors to identify previously unseen objects as unknown and incrementally incorporate them into the set of known categories.<n>Existing frameworks rely heavily on exemplar replay to mitigate catastrophic forgetting, but in some real applications, storing raw data conflicts with data access restrictions and leads to data exposure risks.<n>We propose REAL-OW, a novel rehearsal-free framework that decouples incremental knowledge through a collaborative adapter architecture.
  arXiv  Detail & Related papers  (2026-07-03T06:38:54Z)
• Dual Sparse Aggregation Transformer for Multispectral Object Detection [34.20084627163684]
  We propose a novel Dual Sparse Aggregation Transformer (DSAFormer) for multispectral object detection.<n>Specifically, the DSAFormer is designed to exploit and boost cross-modal complementary information.<n>Our proposed DSAFormer achieves better detection performance against state-of-the-art methods on four public datasets.
  arXiv  Detail & Related papers  (2026-06-30T01:05:24Z)
• HubScan: Detecting Hubness Poisoning in Retrieval-Augmented Generation Systems [0.031068858355080494]
  hubness can be exploited to introduce harmful content, alter search rankings, bypass content filtering, and decrease system performance.<n>We introduce hubscan, an open-source security scanner that evaluates vector indices and embeddings to identify hubs in RAG systems.<n> hubscan achieves 90% recall at a 0.2% alert budget and 100% recall at 0.4%, with adversarial hubs ranking above the 99.8th percentile.
  arXiv  Detail & Related papers  (2026-02-25T21:37:53Z)
• Enhancing Diversity and Feasibility: Joint Population Synthesis from Multi-source Data Using Generative Models [4.73459038844245]
  This study proposes a novel method to simultaneously integrate and synthesize multi-source datasets using a Wasserstein Generative Adversarial Network (WGAN) with gradient penalty.<n>Results show that the proposed joint approach outperforms the sequential baseline, with recall increasing by 7% and precision by 15%.<n>Since synthetic populations serve as a key input for agent-based models (ABM), this multi-source generative approach has the potential to significantly enhance the accuracy and reliability of ABM.
  arXiv  Detail & Related papers  (2026-02-17T00:02:30Z)
• Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaboration [49.9937230730202]
  We propose Search-R2, a novel Actor-Refiner collaboration framework that enhances reasoning through targeted intervention.<n>Our approach decomposes the generation process into an Actor, which produces initial reasoning trajectories.<n>We show that Search-R2 consistently outperforms strong RAG and RL-based baselines across model scales.
  arXiv  Detail & Related papers  (2026-02-03T15:32:09Z)
• CKDA: Cross-modality Knowledge Disentanglement and Alignment for Visible-Infrared Lifelong Person Re-identification [77.07028925223383]
  Lifelong person Re-IDentification aims to match the same person employing continuously collected individual data from different scenarios.<n>To achieve continuous all-day person matching across day and night, Visible-Infrared Lifelong person Re-IDentification (VI-LReID) focuses on sequential training on data from visible and infrared modalities.<n>Existing methods typically exploit cross-modal knowledge distillation to alleviate the catastrophic forgetting of old knowledge.
  arXiv  Detail & Related papers  (2025-11-19T01:30:29Z)
• MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources [113.33902847941941]
  Variance-Aware Sampling (VAS) is a data selection strategy guided by Variance Promotion Score (VPS)<n>We release large-scale, carefully curated resources containing 1.6M long CoT cold-start data and 15k RL QA pairs.<n> Experiments across mathematical reasoning benchmarks demonstrate the effectiveness of both the curated data and the proposed VAS.
  arXiv  Detail & Related papers  (2025-09-25T14:58:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.