From Direction to Magnitude: How Multimodal Instruction-Tuning Reorganizes the Geometric Encoding of Identity-Specifying Prompts in Transformer Hidden States

• URL: http://arxiv.org/abs/2607.09842v1
• Date: Fri, 10 Jul 2026 17:22:36 GMT
• Title: From Direction to Magnitude: How Multimodal Instruction-Tuning Reorganizes the Geometric Encoding of Identity-Specifying Prompts in Transformer Hidden States
• Abstract summary: We investigate whether identity-specifying system prompts produce statistically distinguishable geometric fingerprints in the hidden-state trajectories of four open-weight transformer language models.<n>Three prompt conditions are compared via five metrics, principally the 1-Wasserstein distance between edge-wise of Ollivier-Ricci curvature on kNN trajectory graphs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We investigate whether identity-specifying system prompts produce statistically distinguishable geometric fingerprints in the hidden-state trajectories of four open-weight transformer language models spanning four post-training regimes: no training (Gemma-4-E4B base), multimodal RLHF (Gemma-4-E4B-it), RL distillation (DeepSeek-R1-Distill-Qwen-7B), and SFT (Qwen2.5-7B-Instruct). Three prompt conditions (an identity-specifying axis prompt, a length-matched generic-assistant prompt, and a 26-token vanilla baseline) are compared via five geometric metrics, principally the 1-Wasserstein distance between edge-wise distributions of Ollivier-Ricci curvature on k-NN trajectory graphs. Claims rest on trajectory-level permutation tests with multiple geometric controls (teacher-forced content controls, temporal-chain vs k-NN topology, ABT-projected k-NN, angular vs Euclidean graph construction, B=5000 permutations on borderline statistics). The central finding is a qualitative reorganization of identity encoding across the instruction-tuning boundary: in the base model the fingerprint is direction-coded (separation 0.034, p=0.002 under angular k-NN); in the multimodal instruction-tuned model it migrates into the magnitude (angular separation collapses to p=0.439 while Euclidean survives at p=0.042, and the mean norm of the first generated state inverts its length-ordering, being lowest for the identity prompt). This direction-to-magnitude reorganization is specific to the multimodal instruction-tuning regime, absent under RL distillation and SFT. A teacher-forced control attributes ~30% of the free-running cosine signal to prompt-driven effects. We position W_1 on edge-wise Ollivier-Ricci distributions on k-NN trajectory graphs as a methodological contribution of independent interest.

Related papers

• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• A matched-integrator evaluation of Hamiltonian neural networks on pendulum and Kepler dynamics [0.0]
  Hamiltonian Neural Networks (HNNs) parameterize conservative dynamics through a learned scalar Hamiltonian.<n>We evaluate this prior under a controlled protocol in which an HNN and a parameter-matched feedforward baseline are trained.<n>Results are reported over five independent training seeds.
  arXiv  Detail & Related papers  (2026-08-10T21:12:52Z)
• Eikonal Regularisation in Physics-Informed Neural Networks for Three-Dimensional Level-Set Advection: Transferability of Two-Dimensional Design Principles [0.0]
  We implement an eikonal regulariser to penalise the deviation of $|nabla|$ from unity.<n>We benchmark against a fifth-order WENO solver on identical grids and error measures.<n>We show that the relative $L$ error cannot certify the preservation of thin features.
  arXiv  Detail & Related papers  (2026-08-08T20:20:13Z)
• The Complexity Ceiling Benchmark: A Multi-Domain Evaluation of Sequential Reasoning Under Depth Scaling [8.616356693448987]
  We introduce the Complexity Ceiling Benchmark (CCB), a controlled evaluation of how language-model reasoning decays.<n>CCB fixes the semantic content of a task and varies only its depth N in 5,...,50 across three structurally distinct regimes.<n>We find a consistent pattern of geometric per-step decay with widely separated domain ceilings.
  arXiv  Detail & Related papers  (2026-06-28T08:56:34Z)
• GeoRouteNet: Geometry-Enhanced Non-Autoregressive Neural Solver for the Traveling Salesman Problem [5.202950948929751]
  GeoRouteNet is a geometry-enhanced neural solver for Euclidean traveling salesman problem (TSP)<n>On the model side, GeoRouteNet incorporates centered node features, learnable radial distance basis functions, distance-aware graph attention, LayerNorm-SwiGLU feed-forward blocks, and cross-layer attentive residual encoder mixing.<n>On the training side, we design multi-candidate self-comparison reinforcement learning (MCS-RL), which samples multiple candidate tours per instance.
  arXiv  Detail & Related papers  (2026-06-22T02:31:22Z)
• Trajectory Geometry of Transformer Representations Across Layers [0.5599792629509229]
  We recast the transformer forward pass as a discrete population trajectory through a high-dimensional representation manifold.<n>We characterize trajectory geometry using five metrics computed directly in the ambient space.
  arXiv  Detail & Related papers  (2026-06-08T09:54:07Z)
• Dead Directions: Geometric Singular Learning [0.0]
  Singular learning theory and information geometry have studied the same parameter spaces in mostly separate vocabularies.<n>We bridge them through one primitive, the dead direction: a unit vector along which the Fisher metric degenerates.<n>A selection rule on smooth fibres translates this rate into Watanabe's single-direction contribution to the real log canonical threshold.<n>We then lift this rate to a deep network: a multi-layer K-FAC factorisation writes each Fisher block as a product of activation- and gradient-side rates.
  arXiv  Detail & Related papers  (2026-06-04T09:54:08Z)
• Structure-Informed Estimation for Pilot-Limited MIMO Channels via Tensor Decomposition [51.56484100374058]
  This paper formulates pilot-limited channel estimation as low-rank tensor completion from sparse observations.<n>Experiments on synthetic channels demonstrate 10-20,dB normalized mean-square error (NMSE) improvement over least-squares (LS)<n> evaluations on DeepMIMO ray-tracing channels show 24-44% additional NMSE reduction over pure tensor-based methods.
  arXiv  Detail & Related papers  (2026-02-03T23:38:05Z)
• Variational Bayesian Flow Network for Graph Generation [54.94088904387278]
  We propose Variational Bayesian Flow Network (VBFN) for graph generation.<n>VBFN performs variational lifting to a tractable joint Gaussian variational belief family governed by structured precisions.<n>On synthetic and molecular graph datasets, VBFN improves fidelity and diversity, and surpasses baseline methods.
  arXiv  Detail & Related papers  (2026-01-30T03:59:38Z)
• Scalable Graph Compressed Convolutions [68.85227170390864]
  We propose a differentiable method that applies permutations to calibrate input graphs for Euclidean convolution. Based on the graph calibration, we propose the Compressed Convolution Network (CoCN) for hierarchical graph representation learning.
  arXiv  Detail & Related papers  (2024-07-26T03:14:13Z)
• Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks [54.177130905659155]
  Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks. In this paper, we study a suitable function space for over- parameterized two-layer neural networks with bounded norms.
  arXiv  Detail & Related papers  (2024-04-29T15:04:07Z)
• Relative Pose from SIFT Features [50.81749304115036]
  We derive a new linear constraint relating the unknown elements of the fundamental matrix and the orientation and scale. The proposed constraint is tested on a number of problems in a synthetic environment and on publicly available real-world datasets on more than 80000 image pairs.
  arXiv  Detail & Related papers  (2022-03-15T14:16:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.