Is Inference Mediated by Distinct Semantic Structures in LLMs? A Mechanistic Interpretation

• URL: http://arxiv.org/abs/2605.25520v1
• Date: Mon, 25 May 2026 07:21:07 GMT
• Title: Is Inference Mediated by Distinct Semantic Structures in LLMs? A Mechanistic Interpretation
• Abstract summary: Transformer representations are known to carry label-level information, but whether they encode semantic operations producing those labels is unclear.<n>We investigate this in Natural Language Inference using controlled premise-hypothesis pairs that differ by a single semantic transformation.
• Score: 34.50304918655722
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Predicting a label correctly does not necessarily require representing the operation that produces it. Transformer representations are known to carry label-level information, but whether they encode semantic operations producing those labels is unclear. We investigate this in Natural Language Inference using controlled premise-hypothesis pairs that differ by a single semantic transformation. Using layer-wise activations, we estimate operation-level subspaces via SVD and test their causal relevance through activation steering in four open-weight decoder models. Transformation effects are decodable with $84.8$-$99\%$ accuracy and occupy partially distinct but overlapping subspaces, exceeding random-subspace baselines. Steering experiments show that these directions causally influence predictions, though steerability varies across models; cross-operation steering further reveals structured interference and a dissociation between subspace selectivity and cross-operation independence. These findings indicate that the models encode not only that a hypothesis relates to a premise but also, in part, how it does so, implying that mechanistic analysis and control should operate at the level of semantic operations rather than predicted labels alone.

Related papers

• Look But Don't Touch with Sparse Autoencoders for Unlearning in Diffusion Models [11.366541829206199]
  Sparse autoencoders (SAEs) have been proposed as interpretable tools for concept-level manipulation.<n>We show that while SAEs reliably detect and localize semantic concepts within diffusion model activations, direct intervention in their latent space frequently induces out-of-distribution activations.<n>We use SAE activations purely as semantic detectors to identify image regions containing the target object, and replace those patch embeddings with the ones that do not contain it.
  arXiv  Detail & Related papers  (2026-06-30T14:09:28Z)
• Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift [17.611062308867275]
  We show that distribution shift rotates the subspace that the model actively uses, misaligning the explainer's dictionary trained on in-distribution (ID) activations.<n>We formalize this misalignment as the faithfulness gap, a geometric distance between the ID dictionary and the OOD-active subspace.<n>We propose the Geometry-Adaptive Explainer (GAE), which realigns the explainer's dictionary with the OOD-active subspace while preserving the original feature structure.
  arXiv  Detail & Related papers  (2026-05-21T00:46:01Z)
• Accurate and Efficient Statistical Testing for Word Semantic Breadth [0.0]
  dispersion-based statistics serve as proxies for contextual diversity.<n>We propose a Householder-aligned permutation test to isolate differences from directional differences.<n>Our method reduces Type-I error by 32.5% while preserving sensitivity to genuine breadth differences.
  arXiv  Detail & Related papers  (2026-05-08T17:38:36Z)
• Dissociating Decodability and Causal Use in Bracket-Sequence Transformers [1.8962029954096566]
  We find that depth, distance, and top-of-stack signals are all decodable, yet their causal roles diverge.<n>Results, which extend to a templated natural language setting, suggest that even in a controlled setting, decodability alone does not imply causal use.
  arXiv  Detail & Related papers  (2026-04-24T00:26:34Z)
• Activation Scaling for Steering and Interpreting Language Models [55.59689963561315]
  We argue that successfully intervening on a model is a prerequisite for interpreting its internal workings. We establish a three-term objective: a successful intervention should flip the correct with the wrong token and vice versa. Using gradient-based optimization, this objective lets us learn (and later evaluate) a specific kind of efficient and interpretable intervention.
  arXiv  Detail & Related papers  (2024-10-07T12:01:32Z)
• Adapting to Latent Subgroup Shifts via Concepts and Proxies [82.01141290360562]
  We show that the optimal target predictor can be non-parametrically identified with the help of concept and proxy variables available only in the source domain. For continuous observations, we propose a latent variable model specific to the data generation process at hand.
  arXiv  Detail & Related papers  (2022-12-21T18:30:22Z)
• Confident Sinkhorn Allocation for Pseudo-Labeling [40.883130133661304]
  Semi-supervised learning is a critical tool in reducing machine learning's dependence on labeled data. This paper studies theoretically the role of uncertainty to pseudo-labeling and proposes Confident Sinkhorn Allocation (CSA) CSA identifies the best pseudo-label allocation via optimal transport to only samples with high confidence scores.
  arXiv  Detail & Related papers  (2022-06-13T02:16:26Z)
• Neuro-Symbolic Entropy Regularization [78.16196949641079]
  In structured prediction, the goal is to jointly predict many output variables that together encode a structured object. One approach -- entropy regularization -- posits that decision boundaries should lie in low-probability regions. We propose a loss, neuro-symbolic entropy regularization, that encourages the model to confidently predict a valid object.
  arXiv  Detail & Related papers  (2022-01-25T06:23:10Z)
• Self-training Avoids Using Spurious Features Under Domain Shift [54.794607791641745]
  In unsupervised domain adaptation, conditional entropy minimization and pseudo-labeling work even when the domain shifts are much larger than those analyzed by existing theory. We identify and analyze one particular setting where the domain shift can be large, but certain spurious features correlate with label in the source domain but are independent label in the target.
  arXiv  Detail & Related papers  (2020-06-17T17:51:42Z)
• Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation [49.295165476818866]
  This paper focuses on the unsupervised domain adaptation of transferring the knowledge from the source domain to the target domain in the context of semantic segmentation. Existing approaches usually regard the pseudo label as the ground truth to fully exploit the unlabeled target-domain data. This paper proposes to explicitly estimate the prediction uncertainty during training to rectify the pseudo label learning.
  arXiv  Detail & Related papers  (2020-03-08T12:37:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.