Understanding Knowledge Transfer Mechanism in Heterogeneous MLLM Fusion: A Simple Linear Approach

• URL: http://arxiv.org/abs/2607.26608v1
• Date: Wed, 29 Jul 2026 08:31:27 GMT
• Title: Understanding Knowledge Transfer Mechanism in Heterogeneous MLLM Fusion: A Simple Linear Approach
• Abstract summary: Cross-scale knowledge transfer is determined at first order by capability-dependent responses to a shared injection direction.<n>High-level reasoning gains arise primarily from the language model.<n>These findings recast cross-scale heterogeneous MLLM fusion as selective language-side reasoning transfer within a narrow, low-interference regime.
• Score: 5.040180067606661
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training-free fusion of heterogeneous multimodal large language models (MLLMs) provides a direct route for cross-scale capability transfer, yet improvements in aggregate performance do not reveal what a smaller model actually inherits. Existing studies are largely designed and evaluated on limited task sets or aggregate metrics; as evaluation expands to broader task collections, whether different capabilities can transfer across scales remains poorly understood. To investigate this question, we introduce Cross-Scale Directional Parameter Injection (CDPI), a simple linear probe to analyze cross-scale knowledge transfer during heterogeneous fusion. A local theoretical analysis indicates that knowledge transfer selectivity is determined at first order by capability-dependent responses to a shared injection direction, while second-order curvature effects constrain the effective transfer regime. Across four Qwen3-VL model pairs and twelve multimodal benchmarks, our experiments reveal a consistent pattern of selectivity: gains concentrate on reasoning, particularly high-level reasoning, whereas perception performance remains close to that of the original target model. Component-wise ablations further show that high-level reasoning gains arise primarily from the language model, while ratio analysis finds that positive selective transfer occurs mainly in the small-ratio regime. These findings recast cross-scale heterogeneous MLLM fusion as selective language-side reasoning transfer within a narrow, low-interference regime, rather than broad capability inheritance.

Related papers

• AnE: Pushing the Reasoning Frontier of Multimodal LLMs via Anchor Evolution [61.593935260052795]
  Post-training viaSupervised Fine-Tuning (SFT) and Reinforcement Learning (RL) is crucial for enhancing reasoning in Multimodal Large Language Models (MLLMs)<n>Existing paradigms often reach a performance bottleneck due to the limitations of static data.<n>We propose Anchor Evolution (AnE), a new paradigm that integrates truth-anchored data curation and model evolution.
  arXiv  Detail & Related papers  (2026-05-25T08:26:34Z)
• ES-Merging: Biological MLLM Merging via Embedding Space Signals [52.84455878597969]
  We propose a representation-aware merging framework that estimates merging coefficients from embedding space signals.<n>Our method outperforms existing merging methods and even surpasses task-specific fine-tuned models.
  arXiv  Detail & Related papers  (2026-03-15T14:38:32Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• Mechanistic Data Attribution: Tracing the Training Origins of Interpretable LLM Units [34.05875226612676]
  We introduce Mechanistic Data Attribution (MDA), a scalable framework that employs Influence Functions to trace interpretable units back to specific training samples.<n>We causally validate that targeted intervention--removing or augmenting a small fraction of high-influence samples--significantly modulates the emergence of interpretable heads.
  arXiv  Detail & Related papers  (2026-01-29T17:06:54Z)
• FRISM: Fine-Grained Reasoning Injection via Subspace-Level Model Merging for Vision-Language Models [20.47311573790516]
  We propose FRISM (Fine-grained Reasoning Injection via Subspace-level model Merging), a fine-grained reasoning injection framework based on subspace-level model merging.<n>Experiments demonstrate that FRISM effectively improves reasoning capabilities without compromising the model's original visual capabilities.
  arXiv  Detail & Related papers  (2026-01-29T02:36:19Z)
• Decoupling the Effect of Chain-of-Thought Reasoning: A Human Label Variation Perspective [60.45433515408158]
  We show that long Chain-of-Thought (CoT) serves as a decisive decision-maker for the top option but fails to function as a granular distribution calibrator for ambiguous tasks.<n>We observe a distinct "decoupled mechanism": while CoT improves distributional alignment, final accuracy is dictated by CoT content.
  arXiv  Detail & Related papers  (2026-01-06T16:26:40Z)
• R^2-HGP: A Double-Regularized Gaussian Process for Heterogeneous Transfer Learning [15.278249213859844]
  Multi-output Gaussian process (MGP) models have attracted significant attention for their flexibility and uncertainty-quantification capabilities.<n>They have been widely adopted in multi-source transfer learning scenarios due to their ability to capture inter-task correlations.<n>This paper proposes a Double-Regularized Heterogeneous Gaussian Process framework (R2-HGP) to overcome several challenges in transfer learning.
  arXiv  Detail & Related papers  (2025-12-11T03:38:20Z)
• Enhancing Adversarial Transferability by Balancing Exploration and Exploitation with Gradient-Guided Sampling [82.52485740425321]
  Adrial attacks present a critical challenge to deep neural networks' robustness.<n> transferability of adversarial attacks faces a dilemma between Exploitation (maximizing attack potency) and Exploration (enhancing cross-model generalization)
  arXiv  Detail & Related papers  (2025-11-01T05:43:47Z)
• Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression [7.414126402359073]
  We study the intersection of transfer learning and minimum-$ell$-norm interpolator (MNI) in high-dimensional linear regression.<n>Our research bridges the gap by proposing a novel two-step Transfer MNI approach and analyzing its trade-offs.
  arXiv  Detail & Related papers  (2025-10-17T05:58:16Z)
• Directional Reasoning Injection for Fine-Tuning MLLMs [51.53222423215055]
  Multimodal large language models (MLLMs) are rapidly advancing, yet their reasoning ability often lags behind that of strong text-only counterparts.<n>Existing methods to bridge this gap rely on supervised fine-tuning over large-scale multimodal reasoning data or reinforcement learning.<n>We propose Directional Reasoning Injection for Fine-Tuning (DRIFT) to solve this problem.
  arXiv  Detail & Related papers  (2025-10-16T18:06:46Z)
• CTRLS: Chain-of-Thought Reasoning via Latent State-Transition [57.51370433303236]
  Chain-of-thought (CoT) reasoning enables large language models to break down complex problems into interpretable intermediate steps.<n>We introduce groundingS, a framework that formulates CoT reasoning as a Markov decision process (MDP) with latent state transitions.<n>We show improvements in reasoning accuracy, diversity, and exploration efficiency across benchmark reasoning tasks.
  arXiv  Detail & Related papers  (2025-07-10T21:32:18Z)
• Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferability [23.369773251447636]
  Adversarial examples (AEs) for DNNs have been shown to be transferable.<n>In this paper, we take a further step towards understanding adversarial transferability.
  arXiv  Detail & Related papers  (2023-07-15T19:20:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.