Borrowed Geometry: Computational Reuse of Frozen Text-Pretrained Transformer Weights Across Modalities

• URL: http://arxiv.org/abs/2605.00333v1
• Date: Fri, 01 May 2026 01:23:37 GMT
• Title: Borrowed Geometry: Computational Reuse of Frozen Text-Pretrained Transformer Weights Across Modalities
• Abstract summary: Gemma 4 31B weights pretrained exclusively on text tokens, unmodified, transfer across modality boundaries.<n>Gemma 4 31B is the only model on the small-scale frontier as of April 2026.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Frozen Gemma 4 31B weights pretrained exclusively on text tokens, unmodified, transfer across modality boundaries through a thin trainable interface. (1) OGBench scene-play-singletask-task1-v0: $+4.33$pt over published GCIQL at $n=3$ with std 0.74 -- a published-SOTA win on a robotic manipulation task the substrate has never seen. (2) D4RL Walker2d-medium-v2: Decision-Transformer parity ($76.2 \pm 0.8$, $n=3$) at $0.43\times$ DT's trainable count, with the frozen substrate compressing to a 5L slice ($+1.66$pt over the 6L baseline at $n=3$). (3) Associative recall as the cleanest pretraining-load-bearing case: the frozen slice + a 113K-parameter linear interface reaches L30 best-checkpoint per-bit error 0.0505 ($n=2$); a 6.36M-parameter from-scratch trained transformer at matched capacity ($1/\sqrt{d_k}$ scaling, two seeds, LR sweep) cannot solve the task at all under the protocol (best L30 = 0.4395), an $8.7\times$ advantage. Architecture-alone falsifications: a frozen random transformer with correct $1/\sqrt{d_k}$ scaling stays at random-chance loss for 50k steps; a random-init Gemma slice fails OGBench cube-double-play-task1 entirely (0.89% across $n=3$ where pretrained reaches 60%). A dual-measurement protocol -- text-activation probing on 95 English sentences plus task-ablation on a non-language target -- names individual heads independently identifiable on both protocols: head L26.28 scores $3.7\times$ the slice mean for English token-copying and is the #2 most-critical head for binary copy ablation ($Δ$ L30 $= +0.221$); three further heads (L27.28, L27.2, L27.3) classify by the same protocol. The mechanism is single-model and the cross-modality results are single-task within their respective benchmarks; cross-model replication is structurally constrained because Gemma 4 31B is the only model on the small-scale Pareto frontier as of April 2026.

Related papers

• Certification cost of quantum models: measurement correlation, not parameter count [0.0]
  Certifying an empirical Fisher matrix to relative Frobenius error costs $(B p2 V/(varepsilon2 G))$ circuit executions.<n>Cost-optimal readout weights cut the measured shot budget by $2.67times$ ($1.33$--$4.00$) on hardware.
  arXiv  Detail & Related papers  (2026-09-13T10:47:03Z)
• REFACTOR-VLA: Unsupervised Library Learning of Typed Motor Programs [0.0]
  Most vision-language-action (VLA) models are monolithic.<n>We introduce REFACTOR-VLA, a wake/sleepmodel system for learning reusable skills.
  arXiv  Detail & Related papers  (2026-09-01T13:19:05Z)
• TextNCA: Neural Cellular Automata for Language Modeling via Hierarchical Local Attention [45.024660338672675]
  We define textscTextNCA, a 1D causal windowed-attention realisation of the Neural Cellular Automaton primitive.<n>We study a hierarchical variant that cascades three stages with windows $w in 8, 32, 128$ and $T_s$ shared-weight iterations per stage, all on WikiText-103 at roughly 30M parameters and 60k training steps.
  arXiv  Detail & Related papers  (2026-08-03T10:47:05Z)
• The Security Budget of Code-LLM Prompt Hardening: Provable Limits Under Pass-Only Acceptance [0.0]
  We operationalize the floor as the emphTri-Audit Protocol, a two-axis reporting protocol that separates a prompt-side deductive registry attribute from a model-side empirical corollary.<n>A constrained best-of-family search over deterministic and guarded learned filters on CodeLlama-7B, Qwen2.5-Coder-7B/1.5B and DeepSeek-Coder-6.7B at $n=164$ yields the emphCross-Model Tri-Audit Invariance: of twenty-eight pass-preserving rows, twelve-changed-of-record learned-can
  arXiv  Detail & Related papers  (2026-06-02T08:22:14Z)
• Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasing [51.56484100374058]
  Production LLM applications stack several defense families -- refusal-phrase filters, tokenbudget controls, model allowlists, rate limits, tool-registry authentication -- yetBAS benchmarks report a single aggregate coverage number.<n>We add four-LLM-Top-10-aware agents to a 21-agent baseline scanner and target a lattice of four synthetic LLM endpoints.
  arXiv  Detail & Related papers  (2026-06-01T19:39:25Z)
• Scaling Federated Linear Contextual Bandits via Sketching [49.12000877146222]
  This paper proposes Federated Sketch Contextual Linear Bandits (FSCLB)<n>Experiments on both synthetic and real-world datasets show that FSCLB significantly reduces computational and communication costs by over 90 %.
  arXiv  Detail & Related papers  (2026-05-01T08:22:06Z)
• Copy-as-Decode: Grammar-Constrained Parallel Prefill for LLM Editing [2.6382975801439836]
  LLMs edit text and code by autoregressively regenerating the full output, even when most tokens appear verbatim in the input.<n>We study Copy-as-Decode, a decoding-layer mechanism that recasts edit generation as structured decoding over a two-primitive grammar.
  arXiv  Detail & Related papers  (2026-04-20T12:29:53Z)
• Parity-unfolded distillation architecture for noise-biased platforms [0.0]
  parity-unfolded architecture is a fault-tolerant quantum computing scheme.<n>A state $|Z_krangle = Z1/2k|+rangle$ can be prepared fault-tolerantly using $2k+3 + O(2k/2)$ biased-noise qubits.
  arXiv  Detail & Related papers  (2026-04-16T18:00:32Z)
• Breaking the KV Cache Bottleneck: Fan Duality Model Achieves O(1) Decode Memory with Superior Associative Recall [0.0]
  Fan Duality Model resolves tension between memory efficiency and associative recall in sequence modeling.<n>FDM separates sequence processing into two components: a wave component that compresses long-range patterns into a hidden state, and a particle component that retrieves specific tokens.<n>We propose Freeze-Scan, a two-phase training strategy that freezes the recurrent scan and optimize the cache jointly with embeddings.
  arXiv  Detail & Related papers  (2026-04-09T02:00:30Z)
• Transfer Faster, Price Smarter: Minimax Dynamic Pricing under Cross-Market Preference Shift [8.44638855809941]
  We study contextual dynamic pricing when a target market can leverage K auxiliary markets.<n>We propose Cross-Market Transfer Dynamic Pricing (CM-TDP), the first algorithm that provably handles such model-shift transfer.<n>By bridging transfer learning, robust aggregation, and revenue optimization, CM-TDP moves toward pricing systems that transfer faster, price smarter.
  arXiv  Detail & Related papers  (2025-05-22T18:18:17Z)
• Scalable 3D Registration via Truncated Entry-wise Absolute Residuals [65.04922801371363]
  A $3$D registration approach can process more than ten million ($107$) point pairs with over $99%$ random outliers. We call our method TEAR, as it involves minimizing an outlier-robust loss that computes Truncated Entry-wise Absolute Residuals.
  arXiv  Detail & Related papers  (2024-04-01T04:43:39Z)
• Compressing 1D Time-Channel Separable Convolutions using Sparse Random Ternary Matrices [65.4388266814055]
  We replace 1x1-convolutions in 1D time-channel separable convolutions with constant, sparse random ternary matrices with weights in $-1,0,+1$. For command recognition on Google Speech Commands v1, we improve the state-of-the-art accuracy from $97.21%$ to $97.41%$ at the same network size. For speech recognition on Librispeech, we half the number of weights to be trained while only sacrificing about $1%$ of the floating-point baseline's word error rate.
  arXiv  Detail & Related papers  (2021-03-31T15:09:20Z)
• Provably Efficient Reinforcement Learning with Linear Function Approximation Under Adaptivity Constraints [94.76881135901753]
  We consider two popular limited adaptivity models: batch learning model and rare policy switch model. Our proposed LSVI-UCB-Batch algorithm achieves an $tilde O(sqrtd3H3T + dHT/B)$ regret. For the rare policy switch model, our proposed LSVI-UCB-RareSwitch algorithm enjoys an $tilde O(sqrtd3H3T[1+T/(dH)]dH/B)$ regret.
  arXiv  Detail & Related papers  (2021-01-06T18:56:07Z)
• Deep Learning Meets Projective Clustering [66.726500395069]
  A common approach for compressing NLP networks is to encode the embedding layer as a matrix $AinmathbbRntimes d$. Inspired by emphprojective clustering from computational geometry, we suggest replacing this subspace by a set of $k$ subspaces.
  arXiv  Detail & Related papers  (2020-10-08T22:47:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.