Dead Weights, Live Signals: Feedforward Graphs of Frozen Language Models

• URL: http://arxiv.org/abs/2604.08335v1
• Date: Thu, 09 Apr 2026 15:07:44 GMT
• Title: Dead Weights, Live Signals: Feedforward Graphs of Frozen Language Models
• Abstract summary: We present a feedforward graph architecture in which heterogeneous frozen large language models serve as computational nodes.<n>With only 17.6M trainable parameters against approximately 12B frozen, the architecture achieves 87.3% on ARC-Challenge, 82.8% on OpenBookQA, and 67.2% on MMLU.
• Score: 0.7162422068114824
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a feedforward graph architecture in which heterogeneous frozen large language models serve as computational nodes, communicating through a shared continuous latent space via learned linear projections. Building on recent work demonstrating geometric compatibility between independently trained LLM latent spaces~\cite{armstrong2026thinking}, we extend this finding from static two-model steering to end-to-end trainable multi-node graphs, where projection matrices are optimized jointly via backpropagation through residual stream injection hooks. Three small frozen models (Llama-3.2-1B, Qwen2.5-1.5B, Gemma-2-2B) encode the input into a shared latent space whose aggregate signal is injected into two larger frozen models (Phi-3-mini, Mistral-7B), whose representations feed a lightweight cross-attention output node. With only 17.6M trainable parameters against approximately 12B frozen, the architecture achieves 87.3\% on ARC-Challenge, 82.8\% on OpenBookQA, and 67.2\% on MMLU, outperforming the best single constituent model by 11.4, 6.2, and 1.2 percentage points respectively, and outperforming parameter-matched learned classifiers on frozen single models by 9.1, 5.2, and 6.7 points. Gradient flow through multiple frozen model boundaries is empirically verified to be tractable, and the output node develops selective routing behavior across layer-2 nodes without explicit supervision.

Related papers

• Geometry Without Coordinates: LiDAR Diffusion as a 3D Feature Bridge [0.0]
  We introduce a LiDAR-conditioned diffusion model trained on pseudo-labels from off-the-shelf 2D foundation models.<n>Because the model is conditioned on LiDAR, both its outputs and its intermediate UNet features can be projected back onto the input point cloud.<n>We study this representation directly in point-cloud space, explicitly excluding raw spatial coordinates.
  arXiv  Detail & Related papers  (2026-09-09T15:26:35Z)
• Diffusion Image Generation with Explicit Modeling of Data Manifold Geometry [64.61931485161833]
  Image generative models aim to sample data points from the underlying data manifold.<n>We propose the Data Manifold-aware Image diffusioN moDel (MIND)<n>MIND integrates discrete patch tokenization into the score function of a continuous diffusion model.
  arXiv  Detail & Related papers  (2026-05-25T08:43:14Z)
• Entropy Across the Bridge: Conditional-Marginal Discretization for Flow and Schrödinger Samplers [68.9946633641494]
  Flow matching and Schrdinger bridges define probability, yet their inference grids are usually or inherited from one-endpoint diffusion.<n>We derive a conditional-marginal entropy-rate objective for bridge-aware discretization, separating endpoint-conditioned bridge geometry from marginal flow evolution.<n>On EDM/CIFAR-10, the entropic time-discretization gives the best tested five-step FID (186.3 pm 4.0 versus 200.5 pm 2.9 for linear and 238.0 pm 5.3 for cosine)
  arXiv  Detail & Related papers  (2026-05-15T16:11:10Z)
• Do Foundation Models Know Geometry? Probing Frozen Features for Continuous Physical Measurement [0.0]
  Vision-language models encode geometry that their text pathway fails to express.<n>LoRA fine-tuning (r=16, 2,000 images) narrows this gap to 6.5 degrees.<n>These findings enable a single frozen backbone to function as a multi-task geometric sensor.
  arXiv  Detail & Related papers  (2026-03-06T16:48:27Z)
• Function-Space Decoupled Diffusion for Forward and Inverse Modeling in Carbon Capture and Storage [65.51149575007149]
  We present Fun-DDPS, a generative framework that combines function-space diffusion models with differentiable neural operator surrogates for both forward and inverse modeling.<n>Fun-DDPS produces physically consistent realizations free from the high-frequency artifacts observed in joint-state baselines.
  arXiv  Detail & Related papers  (2026-02-12T18:58:12Z)
• FRIREN: Beyond Trajectories -- A Spectral Lens on Time [1.5939955861266883]
  Long-term time-series forecasting models are often presented as general-purpose solutions that can be applied across domains.<n>We argue that geometric structure is the right abstraction for a dynamic-agnostic foundational model.<n>By connecting modern generative flows with classical spectral analysis, FRIREN makes long-term forecasting both accurate and interpretable.
  arXiv  Detail & Related papers  (2025-05-23T00:52:13Z)
• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free [81.65559031466452]
  We conduct experiments to investigate gating-augmented softmax attention variants.<n>We find that a simple modification-applying a head-specific sigmoid gate after the Scaled Dot-Product Attention (SDPA)-consistently improves performance.
  arXiv  Detail & Related papers  (2025-05-10T17:15:49Z)
• FuseChat-3.0: Preference Optimization Meets Heterogeneous Model Fusion [32.0871035771324]
  We introduce FuseChat-3.0, a suite of large language models (LLMs) developed by integrating the strengths of heterogeneous source LLMs into more compact target LLMs.<n>For target models, we focus on three widely-used smaller variants-Llama-3.1-8B-Instruct, Gemma-2-9B-it, and Qwen-2.5-72B-Instruct.<n>The resulting FuseChat-3.0 models exhibit significant performance gains across tasks such as instruction following, general knowledge, mathematics, and coding.
  arXiv  Detail & Related papers  (2025-03-06T09:03:36Z)
• S*: Test Time Scaling for Code Generation [55.11863577956177]
  We propose S*, the first hybrid test-time scaling framework for code generation.<n>S* substantially improves the coverage and selection accuracy of generated code.
  arXiv  Detail & Related papers  (2025-02-20T09:18:53Z)
• DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling [24.270321913746233]
  We propose a new model merging technique, Drop and rEscaLe via sampLing with mAgnitude (DELLA-Merging), that employs a novel pruning technique, MAGPRUNE. MAGPRUNE first ranks the parameters in order of their magnitude and assigns higher dropout probabilities (p) to parameters with lower ranks corresponding to lower magnitudes.
  arXiv  Detail & Related papers  (2024-06-17T15:02:45Z)
• Prompt Tuning for Parameter-efficient Medical Image Segmentation [79.09285179181225]
  We propose and investigate several contributions to achieve a parameter-efficient but effective adaptation for semantic segmentation on two medical imaging datasets. We pre-train this architecture with a dedicated dense self-supervision scheme based on assignments to online generated prototypes. We demonstrate that the resulting neural network model is able to attenuate the gap between fully fine-tuned and parameter-efficiently adapted models.
  arXiv  Detail & Related papers  (2022-11-16T21:55:05Z)
• Squeezeformer: An Efficient Transformer for Automatic Speech Recognition [99.349598600887]
  Conformer is the de facto backbone model for various downstream speech tasks based on its hybrid attention-convolution architecture. We propose the Squeezeformer model, which consistently outperforms the state-of-the-art ASR models under the same training schemes.
  arXiv  Detail & Related papers  (2022-06-02T06:06:29Z)
• Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition [79.33539539956186]
  We propose a simple method to disentangle multi-scale graph convolutions and a unified spatial-temporal graph convolutional operator named G3D. By coupling these proposals, we develop a powerful feature extractor named MS-G3D based on which our model outperforms previous state-of-the-art methods on three large-scale datasets.
  arXiv  Detail & Related papers  (2020-03-31T11:28:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.