Cross-Trajectory Chimera Interventions Reveal Dissociable Roles of Weight Magnitude and Direction in Grokking

• URL: http://arxiv.org/abs/2607.06628v1
• Date: Tue, 07 Jul 2026 12:12:43 GMT
• Title: Cross-Trajectory Chimera Interventions Reveal Dissociable Roles of Weight Magnitude and Direction in Grokking
• Abstract summary: Single-trajectory interventions show necessity within one run, not portability across runs.<n>We introduce cross-trajectory interventions: given two runs from different seeds, we split each weight vector into a norm and a unit direction, recombine one run's norm with the other's direction, and continue training.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Which properties of a partially trained network are causally portable to a different, independently trained network? Single-trajectory interventions show necessity within one run, not portability across runs. We introduce cross-trajectory chimera interventions: given two runs from different seeds, we split each weight vector into a norm and a unit direction, recombine one run's norm with the other's direction, and continue training. On two modular-arithmetic tasks that grok, the components dissociate. Direction carries a transferable, donor-specific circuit identity: implanting a donor's direction at the recipient's norm drives the run to the donor's circuit in 40/40 cases, while an angle-matched random control yields no shift. The transfer is threshold-like, and its location is predicted by the recipient's norm, separating perfectly by norm class over all 20 pairs (joint permutation probability 1.9e-4). Norm carries only a modest, distributed delay effect and no identity signal. An adaptive bisection procedure localizes the threshold to +/-1/64. Direction indexes which solution a trajectory approaches; norm governs how susceptible that identity is to being overwritten.

Related papers

• Differential Realizability of Static Control Allocation in Multirotors: An Impossibility under Nonredundant Full Actuation and a Pseudoinverse Obstruction under Redundant Actuation [3.317440415766445]
  The signed-quadratic map from physical rotor speed to thrust is not a local diffeomorphism at zero speed.<n>A global fixed-orthant construction establishes existence of regular sections at the cost of persistent task-preserving internal actuation.
  arXiv  Detail & Related papers  (2026-09-11T22:52:19Z)
• Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretability [51.56484100374058]
  We prove the first identifiability theorem for a mechanistic-interpretability primitive.<n>The spectrum is an identifiable, model-intrinsic fingerprint with a stated error bar, not a legible decomposition.
  arXiv  Detail & Related papers  (2026-08-10T19:42:01Z)
• UniCycleFlow: Bidirectional Unpaired Image Translation with a Shared Rectified Flow [14.60117153247222]
  Bidirectional unpaired image translation must preserve source-specific structure while learning coherent transformations in both directions without paired supervision.<n>We propose UniCycleFlow, a rectified-flow framework that represents bidirectional translation as forward and reverse integration of a single time-conditioned velocity field.
  arXiv  Detail & Related papers  (2026-08-07T04:05:04Z)
• Source-Lifted Flow Matching for Intervenable Multimodal Imitation [45.97335144597459]
  Source-Lifted Flow Matching is a source-intervenable flow-matching policy.<n>Handle selects only the source endpoint of the conditional flow, not a mode-specific field.
  arXiv  Detail & Related papers  (2026-07-11T08:33:04Z)
• Actionable Activation Directions for Detecting and Mitigating Emergent Misalignment Across Language Model Families [0.0]
  Fine-tuning language models on insecure code induces emergent misalignment with poorly understood internal structure.<n>We investigate whether this misalignment corresponds to a causally actionable activation-space direction shared across architectures.
  arXiv  Detail & Related papers  (2026-06-18T13:39:59Z)
• GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs [5.018957704056544]
  Activation steering controls model behavior by modifying intermediate hidden states at inference time without retraining.<n>We show that when multiple semantic directions are superposed without constraints, the model collapses.<n>We propose GEMS, a training-free method that maps each source to a corresponding geometric constraint.
  arXiv  Detail & Related papers  (2026-06-18T08:43:33Z)
• Self-Consistent Generative Paths via Admissible Random Variational Transport [14.74009385288423]
  We study a complementary question: when is a generated probability path self-consistent?<n>We define a self-consistent generative path as a random fixed point of admissible local variational transport corrections.<n>The resulting theory turns endpoint matching into path self-consistency testing.
  arXiv  Detail & Related papers  (2026-06-08T02:56:06Z)
• Operator spreading in random circuits with orthogonal or symplectic symmetry [0.0]
  We find that the ensemble-averaged Pauli-string weights relax to a ternary-valued structure, instead of the binary structure of unitary-invariant circuits.<n>For qudit size $q=2$, the butterfly velocity can exceed that of the Haar-random ensemble.
  arXiv  Detail & Related papers  (2026-06-02T17:45:55Z)
• Stochastic Estimation of the Layer-wise Hessian Trace for Monitoring Neural-network Training [45.88028371034407]
  We present a parameter estimator of the trace of the diagonal blocks of the Hessian matrix of the empirical risk of a neural network.<n>The procedure combines the Hutchinson trace estimator with a single Hessian-vector product over the whole parameter vector.<n>We show that correctness under weight sharing requires the layer-wise Hessian to be assembled before the second differentiation.
  arXiv  Detail & Related papers  (2026-05-25T10:24:32Z)
• Transformers Can Learn Rules They've Never Seen: Proof of Computation Beyond Interpolation [0.0]
  We test a strong-only hypothesis in two controlled settings.<n>In Experiment 1, we use a cellular automaton with a pure XOR transition rule.<n>In Experiment 2, we study symbolic operator chains over integers with one operator pair held out.
  arXiv  Detail & Related papers  (2026-03-17T18:02:28Z)
• Branched Schrödinger Bridge Matching [45.105452288011726]
  We introduce Branched Schr"odinger Bridge Matching (BranchSBM), a novel framework that learns branched Schr"odinger bridges.<n>BranchSBM parameterizes multiple time-dependent velocity fields and growth processes, enabling the representation of population-level divergence into terminal distributions.<n>We show that BranchSBM is not only more expressive but also essential for tasks involving multi-path surface navigation, modeling cell fate bifurcations from homogeneous progenitor states, and simulating diverging cellular responses to perturbations.
  arXiv  Detail & Related papers  (2025-06-10T17:29:48Z)
• Proxy Methods for Domain Adaptation [78.03254010884783]
  proxy variables allow for adaptation to distribution shift without explicitly recovering or modeling latent variables. We develop a two-stage kernel estimation approach to adapt to complex distribution shifts in both settings.
  arXiv  Detail & Related papers  (2024-03-12T09:32:41Z)
• Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation [105.22961467028234]
  Skip connections and normalisation layers are ubiquitous for the training of Deep Neural Networks (DNNs) Recent approaches such as Deep Kernel Shaping have made progress towards reducing our reliance on them. But these approaches are incompatible with the self-attention layers present in transformers.
  arXiv  Detail & Related papers  (2023-02-20T21:26:25Z)
• Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation [64.92152574895111]
  We propose a simple Orthogonal Jacobian Regularization (OroJaR) to encourage deep generative model to learn disentangled representations. Our method is effective in disentangled and controllable image generation, and performs favorably against the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-17T15:01:46Z)
• Do Generative Models Know Disentanglement? Contrastive Learning is All You Need [59.033559925639075]
  We propose an unsupervised and model-agnostic method: Disentanglement via Contrast (DisCo) in the Variation Space. DisCo achieves the state-of-the-art disentanglement given pretrained non-disentangled generative models, including GAN, VAE, and Flow.
  arXiv  Detail & Related papers  (2021-02-21T08:01:20Z)
• Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior [54.629850694790036]
  spectral-normalized identity priors (SNIP) is a structured pruning approach that penalizes an entire residual module in a Transformer model toward an identity mapping. We conduct experiments with BERT on 5 GLUE benchmark tasks to demonstrate that SNIP achieves effective pruning results while maintaining comparable performance.
  arXiv  Detail & Related papers  (2020-10-05T05:40:56Z)
• Deriving Differential Target Propagation from Iterating Approximate Inverses [91.3755431537592]
  We show that a particular form of target propagation, relying on learned inverses of each layer, which is differential, gives rise to an update rule which corresponds to an approximate Gauss-Newton gradient-based optimization. We consider several iterative calculations based on local auto-encoders at each layer in order to achieve more precise inversions for more accurate target propagation.
  arXiv  Detail & Related papers  (2020-07-29T22:34:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.