The Geometries of Truth Are Orthogonal Across Tasks

• URL: http://arxiv.org/abs/2506.08572v2
• Date: Fri, 04 Jul 2025 16:21:15 GMT
• Title: The Geometries of Truth Are Orthogonal Across Tasks
• Abstract summary: We show that "geometries of truth" are intrinsically task-dependent and fail to transfer across tasks.<n>More sophisticated approaches fail to overcome this limitation, likely because activation vectors commonly used to classify answers form clearly separated clusters when examined across tasks.
• Score: 24.684427329854202
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) have demonstrated impressive generalization capabilities across various tasks, but their claim to practical relevance is still mired by concerns on their reliability. Recent works have proposed examining the activations produced by an LLM at inference time to assess whether its answer to a question is correct. Some works claim that a "geometry of truth" can be learned from examples, in the sense that the activations that generate correct answers can be distinguished from those leading to mistakes with a linear classifier. In this work, we underline a limitation of these approaches: we observe that these "geometries of truth" are intrinsically task-dependent and fail to transfer across tasks. More precisely, we show that linear classifiers trained across distinct tasks share little similarity and, when trained with sparsity-enforcing regularizers, have almost disjoint supports. We show that more sophisticated approaches (e.g., using mixtures of probes and tasks) fail to overcome this limitation, likely because activation vectors commonly used to classify answers form clearly separated clusters when examined across tasks.

Related papers

• When Correct Demonstrations Hurt: Rethinking the Role of Exemplars in In-Context Learning [14.102553871638243]
  In-context learning (ICL) is often motivated by the intuition that demonstrations help because they provide correct input-output examples.<n>We show that correctness does not guarantee exemplar utility, and some correct demonstrations can even reduce ICL accuracy.<n>Our results show that robust ICL requires evaluating not only whether demonstrations are correct, but also how they influence contextual inference.
  arXiv  Detail & Related papers  (2026-05-25T21:52:49Z)
• Robust and Adaptive Spectral Method for Representation Multi-Task Learning with Contamination [12.976560150719928]
  This paper tackles representation MTL with an unknown and potentially large contamination proportion.<n>We introduce a Robust and Adaptive Spectral method (RAS) that can distill the shared inlier representation effectively and efficiently.
  arXiv  Detail & Related papers  (2025-09-08T11:41:30Z)
• Understanding Task Vectors in In-Context Learning: Emergence, Functionality, and Limitations [19.539276425108987]
  This work proposes the Linear Combination Conjecture, positing that task vectors act as single in-context demonstrations formed through linear combinations of the original ones.<n>We show that task vectors naturally emerge in linear transformers trained on triplet-formatted prompts through loss landscape analysis.<n>We predict the failure of task vectors on representing high-rank mappings and confirm this on practical LLMs.
  arXiv  Detail & Related papers  (2025-06-10T17:59:31Z)
• Out-of-Distribution Generalization of In-Context Learning: A Low-Dimensional Subspace Perspective [9.249642973141107]
  We demystify the out-of-distribution capabilities of in-context learning (ICL) by studying linear regression tasks parameterized with low-rank covariance matrices.<n>We prove that a single-layer linear attention model incurs a test risk with a non-negligible dependence on the angle, illustrating that ICL is not robust to such distribution shifts.<n>This suggests that the OOD generalization ability of Transformers may actually stem from the new task lying within the span of those encountered during training.
  arXiv  Detail & Related papers  (2025-05-20T18:15:49Z)
• Search-Based Correction of Reasoning Chains for Language Models [72.61861891295302]
  Chain-of-Thought (CoT) reasoning has advanced the capabilities and transparency of language models (LMs)<n>We introduce a new self-correction framework that augments each reasoning step in a CoT with a latent variable indicating its veracity.<n>We also introduce Search Corrector, a discrete search algorithm over-valued veracity assignments.
  arXiv  Detail & Related papers  (2025-05-17T04:16:36Z)
• When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers [64.1656365676171]
  Task arithmetic refers to editing the pre-trained model by adding a weighted sum of task vectors.<n>This paper theoretically prove the effectiveness of task addition in simultaneously learning a set of irrelevant or irrelevant tasks.<n>We prove the proper selection for task arithmetic to achieve negation to out-of-domain tasks.
  arXiv  Detail & Related papers  (2025-04-15T08:04:39Z)
• Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyond [62.406687088097605]
  Multi-Task Learning (MTL) is a framework, where multiple related tasks are learned jointly and benefit from a shared representation space. We show that MTL can be successful with classification tasks with little, or non-overlapping annotations. We propose a novel approach, where knowledge exchange is enabled between the tasks via distribution matching.
  arXiv  Detail & Related papers  (2024-01-02T14:18:11Z)
• Is Probing All You Need? Indicator Tasks as an Alternative to Probing Embedding Spaces [19.4968960182412]
  We introduce the term indicator tasks for non-trainable tasks which are used to query embedding spaces for the existence of certain properties. We show that the application of a suitable indicator provides a more accurate picture of the information captured and removed compared to probes.
  arXiv  Detail & Related papers  (2023-10-24T15:08:12Z)
• Nonlinear Meta-Learning Can Guarantee Faster Rates [31.236026714761913]
  We derive theoretical guarantees for meta-learning with nonlinear representations.<n>In particular, we show that additional biases can be mitigated with careful regularization.
  arXiv  Detail & Related papers  (2023-07-20T13:42:13Z)
• Leveraging sparse and shared feature activations for disentangled representation learning [112.22699167017471]
  We propose to leverage knowledge extracted from a diversified set of supervised tasks to learn a common disentangled representation. We validate our approach on six real world distribution shift benchmarks, and different data modalities.
  arXiv  Detail & Related papers  (2023-04-17T01:33:24Z)
• Benign Overfitting in Adversarially Robust Linear Classification [91.42259226639837]
  "Benign overfitting", where classifiers memorize noisy training data yet still achieve a good generalization performance, has drawn great attention in the machine learning community. We show that benign overfitting indeed occurs in adversarial training, a principled approach to defend against adversarial examples.
  arXiv  Detail & Related papers  (2021-12-31T00:27:31Z)
• Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learning [92.05556163518999]
  MARL exacerbates matters by imposing various constraints on communication and observability. For value-based methods, it poses challenges in accurately representing the optimal value function. For policy gradient methods, it makes training the critic difficult and exacerbates the problem of the lagging critic. We show that from a learning theory perspective, both problems can be addressed by accurately representing the associated action-value function.
  arXiv  Detail & Related papers  (2021-05-31T23:08:05Z)
• Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study [75.42182503265056]
  Multi-Task Learning has emerged as a methodology in which multiple tasks are jointly learned by a shared learning algorithm. We deal with heterogeneous MTL, simultaneously addressing detection, classification & regression problems. We build FaceBehaviorNet, the first framework for large-scale face analysis, by jointly learning all facial behavior tasks.
  arXiv  Detail & Related papers  (2021-05-08T22:26:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.