Cross-Family Universality of Behavioral Axes via Anchor-Projected Representations

• URL: http://arxiv.org/abs/2605.09875v1
• Date: Mon, 11 May 2026 02:01:18 GMT
• Title: Cross-Family Universality of Behavioral Axes via Anchor-Projected Representations
• Abstract summary: Large language models from different families use different hidden dimensions, tokenizers, and training procedures.<n>We introduce an anchor-projection framework that maps hidden representations from each model into a shared anchor coordinate space.<n>We find that same-axis directions align tightly across the Llama-Qwen-Mistral-Phi (LQMP) cluster in ACS.
• Score: 8.041250483194434
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models from different families use different hidden dimensions, tokenizers, and training procedures, making behavioral directions difficult to compare or transfer across models. We introduce an anchor-projection framework that maps hidden representations from each model into a shared anchor coordinate space (ACS). Behavioral directions extracted from source models are projected into ACS and averaged into a canonical direction. For a new model, the canonical direction is reconstructed into its native hidden space using only anchor activations, without fine-tuning or target-specific direction extraction. We evaluate five instruction-tuned model families and ten behavioral axes. We find that same-axis directions align tightly across the Llama-Qwen-Mistral-Phi (LQMP) cluster in ACS. This shared structure transfers to downstream tasks. For the aligned LQMP cluster, held-out targets achieve (0.83) ten-way detection accuracy and (0.95) mean binary AUROC, while canonical steering induces refusal-rate shifts of up to +0.46% under distribution shift. Sensitivity analyses show that two source models and small anchor pools already suffice to approximate transferable directions. Overall, ACS provides a novel perspective on cross-family interpretability, revealing that representation-level transfer remains robust across model families.

Related papers

• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• Mapping and Measuring the Behavioral Evolution of Large Language Models [28.603443859513153]
  Benchmark leaderboards summarize how well a language model performs, but not how its behavior relates to that of other models or changes across generations.<n>We characterize the output behavior of 32 models from six families using their responses to a shared bank of 10,000 prompts.
  arXiv  Detail & Related papers  (2026-08-11T15:07:28Z)
• Improving Relative Representations with Learned Anchors and Whitened Inner Products [0.0]
  Independently trained neural models typically converge to incompatible latent representations.<n>Relative Representations (RR) address this by mapping absolute coordinates to a shared space defined by similarities to common anchor points.<n>Traditional implementations rely on randomly sampled anchors and cosine similarity.<n>We propose a robust framework for cross-model communication based on two improvements.
  arXiv  Detail & Related papers  (2026-05-28T21:43:53Z)
• CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning [7.180013165859104]
  CRAFT is a continual learning framework that avoids updating model weights.<n>It first routes each task to a group of similar tasks based on output-distribution divergence.<n>It then fine-tunes the model using a Kullback-Leibler (KL) divergence against the group's prior state.<n>Finally, it merges interventions for the updated task into the shared representation using the same KL signal.
  arXiv  Detail & Related papers  (2026-05-07T06:24:13Z)
• ATLAS: Constitution-Conditioned Latent Geometry and Redistribution Across Language Models and Neural Perturbation Data [0.0]
  Constitution-conditioned post-training can be analysed as a structured perturbation of a model's learned representational geometry.<n>We introduce ATLAS, a geometry-first program that traces constitution-induced hidden-state structure across charts, models, and substrates.
  arXiv  Detail & Related papers  (2026-04-19T23:26:02Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• Concept Heterogeneity-aware Representation Steering [6.247724933679344]
  Representation steering offers a mechanism for controlling the behavior of large language models (LLMs) by intervening on internal activations at inference time.<n>Most existing methods rely on a single global steering direction, typically obtained via difference-in-means over contrastive datasets.<n>In this work, we view representation steering through the lens of optimal transport (OT), noting that standard difference-in-means steering implicitly corresponds to the OT map between two unimodal Gaussian distributions.
  arXiv  Detail & Related papers  (2026-02-13T11:07:23Z)
• Model Specific Task Similarity for Vision Language Model Selection via Layer Conductance [92.72779885657373]
  We propose a framework that grounds model selection in the internal functional dynamics of the visual encoder.<n>Our approach represents each task via layer wise conductance and derives a target-conditioned block importance distribution through entropy regularized alignment.<n>Building on this, we introduce Directional Conductance Divergence (DCD), an asymmetric metric that quantifies how effectively a source task covers the target's salient functional blocks.
  arXiv  Detail & Related papers  (2026-02-01T17:29:43Z)
• Learning Cross-view Geo-localization Embeddings via Dynamic Weighted Decorrelation Regularization [52.493240055559916]
  Cross-view geo-localization aims to spot images of the same location shot from two platforms, e.g., the drone platform and the satellite platform. Existing methods usually focus on optimizing the distance between one embedding with others in the feature space. In this paper, we argue that the low redundancy is also of importance, which motivates the model to mine more diverse patterns.
  arXiv  Detail & Related papers  (2022-11-10T02:13:10Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• Do Generative Models Know Disentanglement? Contrastive Learning is All You Need [59.033559925639075]
  We propose an unsupervised and model-agnostic method: Disentanglement via Contrast (DisCo) in the Variation Space. DisCo achieves the state-of-the-art disentanglement given pretrained non-disentangled generative models, including GAN, VAE, and Flow.
  arXiv  Detail & Related papers  (2021-02-21T08:01:20Z)
• Channelized Axial Attention for Semantic Segmentation [70.14921019774793]
  We propose the Channelized Axial Attention (CAA) to seamlessly integratechannel attention and axial attention with reduced computationalcomplexity. Our CAA not onlyrequires much less computation resources compared with otherdual attention models such as DANet, but also outperforms the state-of-the-art ResNet-101-based segmentation models on alltested datasets.
  arXiv  Detail & Related papers  (2021-01-19T03:08:03Z)
• Scope Head for Accurate Localization in Object Detection [135.9979405835606]
  We propose a novel detector coined as ScopeNet, which models anchors of each location as a mutually dependent relationship. With our concise and effective design, the proposed ScopeNet achieves state-of-the-art results on COCO.
  arXiv  Detail & Related papers  (2020-05-11T04:00:09Z)
• From Anchor Generation to Distribution Alignment: Learning a Discriminative Embedding Space for Zero-Shot Recognition [46.47620562161315]
  In zero-shot learning (ZSL), the samples to be classified are usually projected into side information templates such as attributes. We propose a novel framework called Discriminative Anchor Generation and Distribution Alignment Model (DAGDA) Firstly, in order to rectify the distribution of original templates, a diffusion based graph convolutional network, which can explicitly model the interaction between class and side information, is proposed to produce discriminative anchors. Secondly, to further align the samples with the corresponding anchors in anchor space, which aims to refine the distribution in a fine-grained manner, we introduce a semantic relation regularization
  arXiv  Detail & Related papers  (2020-02-10T05:25:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.