Probing for Representation Manifolds in Superposition

• URL: http://arxiv.org/abs/2605.18537v1
• Date: Mon, 18 May 2026 15:20:35 GMT
• Title: Probing for Representation Manifolds in Superposition
• Abstract summary: This paper introduces the Manifold Probe, a supervised method for discovering representation manifold in superposition.<n>We demonstrate the probe on representations of time and space in Llama 2-7b.<n>In the case of time, we show that by steering along the manifold, we can influence the model's completions about the years in which famous songs, movies and books were released.
• Score: 55.18010930078062
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces the Manifold Probe, a supervised method for discovering representation manifolds in superposition. The method generalizes linear regression probes by learning the space of features of a concept that can be linearly predicted from the representations, and then learning the directions used to encode them. We demonstrate the probe on representations of time and space in Llama 2-7b, finding manifolds which linearly represent an interpretable set of features in each case. In the case of time, we show that by steering along the manifold, we can influence the model's completions about the years in which famous songs, movies and books were released, providing evidence that the Manifold Probe can discover manifolds which are causally involved in model behaviour.

Related papers

• The Shape of Beliefs: Geometry, Dynamics, and Interventions along Representation Manifolds of Language Models' Posteriors [24.477029700560113]
  Large language models (LLMs) represent prompt-conditioned beliefs (posteriors over answers and claims)<n>We study a controlled setting in which Llama-3.2 generates samples from a normal distribution by implicitly inferring its parameters.<n>We find representations of curved "belief manifold" for these parameters form with sufficient in-context learning.
  arXiv  Detail & Related papers  (2026-02-02T16:45:05Z)
• Riemannian Denoising Diffusion Probabilistic Models [7.964790563398277]
  We propose RDDPMs for learning distributions on submanifolds of Euclidean space that are level sets of functions.<n>We provide a theoretical analysis of our method in the continuous-time limit.<n>The capability of our method is demonstrated on datasets from previous studies and on new sampled datasets.
  arXiv  Detail & Related papers  (2025-05-07T11:37:16Z)
• Quantifying Manifolds: Do the manifolds learned by Generative Adversarial Networks converge to the real data manifold [1.9186271764628762]
  We study the dimensions and topological features of the intrinsic manifold learned by the ML model, how these metrics change as we continue to train the model, and whether these metrics convergence over the course of training to the metrics of the real data manifold.
  arXiv  Detail & Related papers  (2024-03-08T04:23:50Z)
• Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference [110.47649327040392]
  Given time series data, how can we answer questions like "what will happen in the future?" and "how did we get here?"<n>We show how these questions can have compact, closed form solutions in terms of learned representations.
  arXiv  Detail & Related papers  (2024-03-06T22:27:30Z)
• On the Origins of Linear Representations in Large Language Models [51.88404605700344]
  We introduce a simple latent variable model to formalize the concept dynamics of the next token prediction. Experiments show that linear representations emerge when learning from data matching the latent variable model. We additionally confirm some predictions of the theory using the LLaMA-2 large language model.
  arXiv  Detail & Related papers  (2024-03-06T17:17:36Z)
• A Heat Diffusion Perspective on Geodesic Preserving Dimensionality Reduction [66.21060114843202]
  We propose a more general heat kernel based manifold embedding method that we call heat geodesic embeddings. Results show that our method outperforms existing state of the art in preserving ground truth manifold distances. We also showcase our method on single cell RNA-sequencing datasets with both continuum and cluster structure.
  arXiv  Detail & Related papers  (2023-05-30T13:58:50Z)
• Convolutional Filtering on Sampled Manifolds [122.06927400759021]
  We show that convolutional filtering on a sampled manifold converges to continuous manifold filtering. Our findings are further demonstrated empirically on a problem of navigation control.
  arXiv  Detail & Related papers  (2022-11-20T19:09:50Z)
• The Manifold Scattering Transform for High-Dimensional Point Cloud Data [16.500568323161563]
  We present practical schemes for implementing the manifold scattering transform to datasets arising in naturalistic systems. We show that our methods are effective for signal classification and manifold classification tasks.
  arXiv  Detail & Related papers  (2022-06-21T02:15:00Z)
• Sample complexity and effective dimension for regression on manifolds [13.774258153124205]
  We consider the theory of regression on a manifold using kernel reproducing Hilbert space methods. We show that certain spaces of smooth functions on a manifold are effectively finite-dimensional, with a complexity that scales according to the manifold dimension.
  arXiv  Detail & Related papers  (2020-06-13T14:09:55Z)
• Evaluating the Disentanglement of Deep Generative Models through Manifold Topology [66.06153115971732]
  We present a method for quantifying disentanglement that only uses the generative model. We empirically evaluate several state-of-the-art models across multiple datasets.
  arXiv  Detail & Related papers  (2020-06-05T20:54:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.