Uncovering Uncertainty in Transformer Inference

• URL: http://arxiv.org/abs/2412.05768v1
• Date: Sun, 08 Dec 2024 00:46:10 GMT
• Title: Uncovering Uncertainty in Transformer Inference
• Abstract summary: We explore the Iterative Inference Hypothesis (IIH) within the context of transformer-based language models.<n>Our findings provide empirical support for the IIH, showing that the nth token embedding in the residual stream follows a trajectory of decreasing loss.<n>We introduce a method utilizing cross-entropy to detect this uncertainty and demonstrate its potential to distinguish between correct and incorrect token generations on a dataset of idioms.
• Score: 1.1726838461147573
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We explore the Iterative Inference Hypothesis (IIH) within the context of transformer-based language models, aiming to understand how a model's latent representations are progressively refined and whether observable differences are present between correct and incorrect generations. Our findings provide empirical support for the IIH, showing that the nth token embedding in the residual stream follows a trajectory of decreasing loss. Additionally, we observe that the rate at which residual embeddings converge to a stable output representation reflects uncertainty in the token generation process. Finally, we introduce a method utilizing cross-entropy to detect this uncertainty and demonstrate its potential to distinguish between correct and incorrect token generations on a dataset of idioms.

Related papers

• Disentangling Continuous-Time Latent Dynamics: Identifiability of Latent SDEs via Diffusion Shifts [60.95873409673775]
  identifiability in continuous-time latent differential equation models remains largely open.<n>We study additive-noise latent SDEs observed through an unknown nonlinear diffeomorphism.<n>We propose a two-stage estimator for latent disentanglement and optional graph recovery.
  arXiv  Detail & Related papers  (2026-06-26T16:18:28Z)
• The Loss Is Not Enough: Sampling Conditions and Inductive Bias in Contrastive Representation Learning [2.575874871258672]
  We develop a measure-theoretic framework formalizing the diversity condition.<n>We show that the standard full-support von Mises-Fisher setting implies the satisfaction of the diversity condition.<n>Our results clarify how sampling mechanisms and inductive bias interact in contrastive representation learning.
  arXiv  Detail & Related papers  (2026-06-02T23:08:30Z)
• An Isotropic Approach to Efficient Uncertainty Quantification with Gradient Norms [7.439615448864818]
  We propose a lightweight method for quantifying predictive uncertainty in neural networks.<n>We use two approximations: a first-order Taylor expansion that expresses uncertainty in terms of the gradient of the prediction, and an isotropy assumption on the parameter covariance.<n>We then use the estimates to investigate when each uncertainty type carries useful signal for predicting answer correctness in question answering with large language models.
  arXiv  Detail & Related papers  (2026-03-31T09:13:09Z)
• Disentangled Representation Learning via Flow Matching [48.12507436294143]
  Disentangled representation learning aims to capture the underlying explanatory factors of observed data.<n>Existing diffusion-based methods encourage factor independence via inductive biases, yet frequently lack strong semantic alignment.<n>We propose a flow matching-based framework for disentangled representation learning, which casts disentanglement as learning factor-conditioned flows in a compact latent space.
  arXiv  Detail & Related papers  (2026-02-05T02:14:36Z)
• Representation Learning for Distributional Perturbation Extrapolation [14.951777123174082]
  We consider the problem of modelling the effects of unseen perturbations on low-level measurements such as RNA sequencing data. We propose a new method, the perturbation distribution autoencoder (PDAE), which is trained by maximising the distributional similarity between true and predicted perturbation distributions.
  arXiv  Detail & Related papers  (2025-04-25T17:44:04Z)
• PseudoNeg-MAE: Self-Supervised Point Cloud Learning using Conditional Pseudo-Negative Embeddings [55.55445978692678]
  PseudoNeg-MAE enhances global feature representation of point cloud masked autoencoders by making them both discriminative and sensitive to transformations.<n>We propose a novel loss that explicitly penalizes invariant collapse, enabling the network to capture richer transformation cues while preserving discriminative representations.
  arXiv  Detail & Related papers  (2024-09-24T07:57:21Z)
• Identifiable Latent Neural Causal Models [82.14087963690561]
  Causal representation learning seeks to uncover latent, high-level causal representations from low-level observed data. We determine the types of distribution shifts that do contribute to the identifiability of causal representations. We translate our findings into a practical algorithm, allowing for the acquisition of reliable latent causal representations.
  arXiv  Detail & Related papers  (2024-03-23T04:13:55Z)
• From Registration Uncertainty to Segmentation Uncertainty [11.294691606431526]
  We propose a novel framework to concurrently estimate both the epistemic and aleatoric segmentation uncertainties for image registration. By introducing segmentation uncertainty along with existing methods for estimating registration uncertainty, we offer vital insights into the potential uncertainties at different stages of image registration.
  arXiv  Detail & Related papers  (2024-03-08T07:16:14Z)
• Identifiable Latent Polynomial Causal Models Through the Lens of Change [82.14087963690561]
  Causal representation learning aims to unveil latent high-level causal representations from observed low-level data. One of its primary tasks is to provide reliable assurance of identifying these latent causal models, known as identifiability.
  arXiv  Detail & Related papers  (2023-10-24T07:46:10Z)
• Learning Disentangled Discrete Representations [22.5004558029479]
  We show the relationship between discrete latent spaces and disentangled representations by replacing the standard Gaussian variational autoencoder with a tailored categorical variational autoencoder. We provide both analytical and empirical findings that demonstrate the advantages of discrete VAEs for learning disentangled representations.
  arXiv  Detail & Related papers  (2023-07-26T12:29:58Z)
• Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processes [52.92110730286403]
  It is commonly believed that the marginal likelihood should be reminiscent of cross-validation metrics and that both should deteriorate with larger input dimensions. We prove that by tuning hyper parameters, the performance, as measured by the marginal likelihood, improves monotonically with the input dimension. We also prove that cross-validation metrics exhibit qualitatively different behavior that is characteristic of double descent.
  arXiv  Detail & Related papers  (2022-10-14T08:09:33Z)
• Identifying Weight-Variant Latent Causal Models [82.14087963690561]
  We find that transitivity acts as a key role in impeding the identifiability of latent causal representations. Under some mild assumptions, we can show that the latent causal representations can be identified up to trivial permutation and scaling. We propose a novel method, termed Structural caUsAl Variational autoEncoder, which directly learns latent causal representations and causal relationships among them.
  arXiv  Detail & Related papers  (2022-08-30T11:12:59Z)
• Which Invariance Should We Transfer? A Causal Minimax Learning Approach [18.71316951734806]
  We present a comprehensive minimax analysis from a causal perspective. We propose an efficient algorithm to search for the subset with minimal worst-case risk. The effectiveness and efficiency of our methods are demonstrated on synthetic data and the diagnosis of Alzheimer's disease.
  arXiv  Detail & Related papers  (2021-07-05T09:07:29Z)
• The Hidden Uncertainty in a Neural Networks Activations [105.4223982696279]
  The distribution of a neural network's latent representations has been successfully used to detect out-of-distribution (OOD) data. This work investigates whether this distribution correlates with a model's epistemic uncertainty, thus indicating its ability to generalise to novel inputs.
  arXiv  Detail & Related papers  (2020-12-05T17:30:35Z)
• Learning Disentangled Representations with Latent Variation Predictability [102.4163768995288]
  This paper defines the variation predictability of latent disentangled representations. Within an adversarial generation process, we encourage variation predictability by maximizing the mutual information between latent variations and corresponding image pairs. We develop an evaluation metric that does not rely on the ground-truth generative factors to measure the disentanglement of latent representations.
  arXiv  Detail & Related papers  (2020-07-25T08:54:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.