Encoded Early, Used Late: Where Transformers Begin to Act on an Inferred Partner's Expertise

• URL: http://arxiv.org/abs/2609.07139v1
• Date: Mon, 07 Sep 2026 07:41:57 GMT
• Title: Encoded Early, Used Late: Where Transformers Begin to Act on an Inferred Partner's Expertise
• Abstract summary: We show that a transformer can make an attribute linearly decodable in its residual stream at a depth where that attribute does not yet influence the output.<n>We ask whether it also holds for an attribute the model must infer gradually over a conversation, namely how expert its dialogue partner is.
• Score: 5.269026412088572
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A transformer can make an attribute linearly decodable in its residual stream at a depth where that attribute does not yet influence the output. This gap between where information is readable and where it is used has been shown for attributes stated directly in the input. We ask whether it also holds for an attribute the model must infer gradually over a conversation, namely how expert its dialogue partner is. Using ExpertCollab, a corpus of multi-turn research-planning dialogues between model-played personas at four expertise levels, we find that partner expertise is most decodable in the early layers and falls to near chance before the midpoint of the network. Counterfactual patching shows that injecting the expertise difference at the layer of peak decodability barely changes a fixed late-layer readout, whereas the same difference injected past the midpoint propagates almost completely, a separation of more than an order of magnitude. A content-matched random control and a probe-free diagnostic place the transition at the same early layer, and a statically specified control attribute stays decodable throughout. An inferred relational attribute is therefore represented well before it becomes causally active, which bounds where any attempt to read out or steer partner-conditioned behavior must intervene. We use one model on a synthetic corpus as an initial demonstration.

Related papers

• Bypass Observation: A Conceptual Design of a Non-Intrusive Layer-Wise Semantic Extraction Architecture [2.8943395720203386]
  We introduce Bypass Observation, a non-intrusive layer-wise readout architecture.<n>We derive a closed-form overhead approximation governed primarily by V/(12d)<n>We argue that Bypass Observation can make model iteration more observable while remaining only a partial, potentially misleading projection of hidden states.
  arXiv  Detail & Related papers  (2026-09-12T08:35:45Z)
• Knowledge-Guided Multimodal Reasoning over Interacting Streams for Video-Level Ambivalence and Hesitancy Recognition [0.9297355862757838]
  PRISM-AH is a framework that treats A/H as a multimodal conflict that unfolds over time.<n>A knowledge-guided large language model then reasons over structured evidence using the expert cue taxonomy of the dataset.<n>On the labelled public test partition of 525 videos, PRISM-AH attains a macro F1 of 0.6133, compared to the reported zero-shot baseline of 0.2827.
  arXiv  Detail & Related papers  (2026-07-28T16:44:38Z)
• Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steering [87.43777061308658]
  We analyze predictive behavior in full spoken language models (FDSLMs)<n>We show that FDSLMs dynamically modulate their internal focus between two states: a generative state aligned with model output generation and a perceptive state aligned incoming user input.<n>During user interruptions, the model remains transiently biased toward the generative state before transitioning into the perceptive state, causing it miss the beginning of incoming input.
  arXiv  Detail & Related papers  (2026-06-09T19:08:07Z)
• Back to Source: Open-Set Continual Test-Time Adaptation via Domain Compensation [18.70132691409063]
  Test-Time Adaptation aims to mitigate distributional shifts between training and test domains during inference time.<n>The coupling of domain and semantic shifts often collapses the feature space, severely degrading both classification and out-of-distribution detection.<n>We propose DOmain COmpensation (DOCO), a lightweight and effective framework that robustly performs domain adaptation and OOD detection in a synergistic, closed loop.
  arXiv  Detail & Related papers  (2026-04-23T15:29:29Z)
• Early stopping by correlating online indicators in neural networks [0.24578723416255746]
  We propose a novel technique to identify overfitting phenomena when training the learner. Our proposal exploits the correlation over time in a collection of online indicators. As opposed to previous approaches focused on a single criterion, we take advantage of subsidiarities between independent assessments.
  arXiv  Detail & Related papers  (2024-02-04T14:57:20Z)
• Exploring Diffusion Time-steps for Unsupervised Representation Learning [72.43246871893936]
  We build a theoretical framework that connects the diffusion time-steps and the hidden attributes. On CelebA, FFHQ, and Bedroom datasets, the learned feature significantly improves classification.
  arXiv  Detail & Related papers  (2024-01-21T08:35:25Z)
• Don't Miss Out on Novelty: Importance of Novel Features for Deep Anomaly Detection [64.21963650519312]
  Anomaly Detection (AD) is a critical task that involves identifying observations that do not conform to a learned model of normality. We propose a novel approach to AD using explainability to capture such novel features as unexplained observations in the input space. Our approach establishes a new state-of-the-art across multiple benchmarks, handling diverse anomaly types.
  arXiv  Detail & Related papers  (2023-10-01T21:24:05Z)
• Context-aware attention layers coupled with optimal transport domain adaptation and multimodal fusion methods for recognizing dementia from spontaneous speech [0.0]
  Alzheimer's disease (AD) constitutes a complex neurocognitive disease and is the main cause of dementia. We propose some new methods for detecting AD patients, which capture the intra- and cross-modal interactions. Experiments conducted on the ADReSS and ADReSSo Challenge indicate the efficacy of our introduced approaches over existing research initiatives.
  arXiv  Detail & Related papers  (2023-05-25T18:18:09Z)
• Causal Effect Estimation with Variational AutoEncoder and the Front Door Criterion [23.20371860838245]
  The front-door criterion is often difficult to identify the set of variables used for front-door adjustment from data. By leveraging the ability of deep generative models in representation learning, we propose FDVAE to learn the representation of a Front-Door adjustment set with a Variational AutoEncoder.
  arXiv  Detail & Related papers  (2023-04-24T10:04:28Z)
• DOAD: Decoupled One Stage Action Detection Network [77.14883592642782]
  Localizing people and recognizing their actions from videos is a challenging task towards high-level video understanding. Existing methods are mostly two-stage based, with one stage for person bounding box generation and the other stage for action recognition. We present a decoupled one-stage network dubbed DOAD, to improve the efficiency for-temporal action detection.
  arXiv  Detail & Related papers  (2023-04-01T08:06:43Z)
• Improving Out-of-Distribution Generalization of Neural Rerankers with Contextualized Late Interaction [52.63663547523033]
  Late interaction, the simplest form of multi-vector, is also helpful to neural rerankers that only use the [] vector to compute the similarity score. We show that the finding is consistent across different model sizes and first-stage retrievers of diverse natures.
  arXiv  Detail & Related papers  (2023-02-13T18:42:17Z)
• Is Disentanglement enough? On Latent Representations for Controllable Music Generation [78.8942067357231]
  In the absence of a strong generative decoder, disentanglement does not necessarily imply controllability. The structure of the latent space with respect to the VAE-decoder plays an important role in boosting the ability of a generative model to manipulate different attributes.
  arXiv  Detail & Related papers  (2021-08-01T18:37:43Z)
• Self-Attention Attribution: Interpreting Information Interactions Inside Transformer [89.21584915290319]
  We propose a self-attention attribution method to interpret the information interactions inside Transformer. We show that the attribution results can be used as adversarial patterns to implement non-targeted attacks towards BERT.
  arXiv  Detail & Related papers  (2020-04-23T14:58:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.