Riemannian-Manifold Steering: Geometry-Aware Generative Autoencoders for Label-Free Steering

• URL: http://arxiv.org/abs/2605.24942v1
• Date: Sun, 24 May 2026 08:41:56 GMT
• Title: Riemannian-Manifold Steering: Geometry-Aware Generative Autoencoders for Label-Free Steering
• Abstract summary: We recast manifold steering as textbfRiemannian geodesic computation on activation space.<n>We approximate this with a learned encoder trained on output distances over a small concept-token schema.
• Score: 3.854000879223449
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Steering a language model - intervening on its internal activations to change downstream behaviour - has recently expanded beyond linear interpolation to nonlinear methods such as angular and kernelized steering, which define intervention transformations without learning an explicit geometry over paths in activation space. Freshly introduced geometry-aware manifold methods do learn such a geometry, but require labelled class centroids together with prescribed cyclic or sequential structure. These assumptions restrict where manifold steering can be applied, since existing constructions require labelled centroids and compatible boundary conditions. We recast manifold steering more broadly as \textbf{Riemannian geodesic computation} on activation space, recovering linear and labelled-spline steering as geodesics under particular choices of metric. A principled metric within this framework is the output-space Hellinger distance pulled back to activations; we approximate this with a learned encoder trained on output distances over a small concept-token schema - no per-prompt labels, no topology prior, and no per-task curve fitting. Empirically, the method reliably drives the model onto the target class across all tasks in a standard four-task language-model arithmetic benchmark, while following more behaviourally natural trajectories than baselines on smaller output spaces. We thereby provide a unified Riemannian framework for manifold steering together with a schema-supervised, label-free instantiation that operates without labelled centroids or prescribed boundary conditions.

Related papers

• Landmark shape spaces with induced metrics [2.1665689529884697]
  We present a unification of Kendall's landmark shape spaces, where rigid motions are factored out and scale fixed on landmark configurations equipped with Euclidean geometry.<n>The regularity of the descending metric prevents landmarks from colliding, the metric is defined in the ambient space independent of the number of landmarks, local rigid transformations are preserved, global rigid motions are removed, and scale fixed.
  arXiv  Detail & Related papers  (2026-07-30T11:41:19Z)
• AutoPath: Learning Transferable Goal-Conditioned Stochastic Path Prior for Safe Navigation Without Human Demonstrations [103.61864552260316]
  Real-time navigation in dynamic environments requires collision-free and dynamically feasible motion under limited perception.<n>We formulate navigation as learning a transferable goal-aligned path prior that models a reusable distribution over goal-aligned geometry-consistent local paths conditioned on local observations.<n>We develop a structured prior learning framework that parameterizes local paths using a geometry-aware polar action manifold and incorporates risk-sensitive utility shaping with multi-goal distributional rollouts for stable and safety-aware planning.
  arXiv  Detail & Related papers  (2026-07-13T16:00:41Z)
• Geometry-Anchored Transport Framework for Exemplar-Free Class-Incremental Learning [6.63181149575886]
  We present the Geometry-Anchored Transport Framework to mitigate macroscopic anisotropic drift.<n>We also introduce a Topology-Aware Evolution objective that regularizes localized manifold degradation.<n> Experiments across CIFAR-100, TinyImageNet, and ImageNet-100 demonstrate that the proposed framework consistently improves upon existing post-hoc alternatives.
  arXiv  Detail & Related papers  (2026-06-24T03:27:20Z)
• A Mean Curvature Approach to Boundary Detection: Geometric Insights for Unsupervised Learning [52.452902154360565]
  We introduce Mean Curvature Boundary Points (MCBP), a novel geometric framework grounded in Geometric Machine Learning.<n>MCBP explicitly models the intrinsic curvature of the data manifold and computes pointwise mean curvature without requiring principled manifold parametrization.<n>Experiments on synthetic and real-world datasets demonstrate that MCBP consistently improves clustering performance.
  arXiv  Detail & Related papers  (2026-05-05T20:19:09Z)
• Directional Mollification for Controlled Smooth Path Generation [2.6381163133447836]
  Mollification has been recently proposed as a computationally efficient and analytically tractable tool for path generation.<n>We introduce directional mollification, a novel operator that resolves this limitation while retaining the analytical tractability of classical mollification.
  arXiv  Detail & Related papers  (2026-03-23T11:18:43Z)
• Parallelised Differentiable Straightest Geodesics for 3D Meshes [47.61846725512702]
  We present a principled framework to compute the exponential map on Riemannian surfaces discretised as meshes is straightest geodesics.<n>We show how our differentiable exponential map can improve learning and optimisation pipelines on general geometries.
  arXiv  Detail & Related papers  (2026-03-16T18:10:28Z)
• Curveball Steering: The Right Direction To Steer Isn't Always Linear [7.264147948470562]
  Activation steering is a widely used approach for controlling large language model (LLM) behavior by intervening on internal representations.<n>We propose "Curveball steering", a nonlinear steering method based on PCA kernel that performs interventions in a feature space.
  arXiv  Detail & Related papers  (2026-03-10T07:45:35Z)
• Riemannian Flow Matching for Disentangled Graph Domain Adaptation [51.98961391065951]
  Graph Domain Adaptation (GDA) typically uses adversarial learning to align graph embeddings in Euclidean space.<n>DisRFM is a geometry-aware GDA framework that unifies embedding and flow-based transport.
  arXiv  Detail & Related papers  (2026-01-31T11:05:35Z)
• Enforcing Latent Euclidean Geometry in Single-Cell VAEs for Manifold Interpolation [79.27003481818413]
  We introduce FlatVI, a training framework that regularises the latent manifold of discrete-likelihood variational autoencoders towards Euclidean geometry.<n>By encouraging straight lines in the latent space to approximate geodesics on the decoded single-cell manifold, FlatVI enhances compatibility with downstream approaches.
  arXiv  Detail & Related papers  (2025-07-15T23:08:14Z)
• Thinner Latent Spaces: Detecting Dimension and Imposing Invariance with Conformal Autoencoders [8.743941823307967]
  We show that orthogonality relations within the latent layer of the network can be leveraged to infer the intrinsic dimensionality of nonlinear manifold data sets.<n>We outline the relevant theory relying on differential geometry, and describe the corresponding gradient-descent optimization algorithm.
  arXiv  Detail & Related papers  (2024-08-28T20:56:35Z)
• Curve Your Attention: Mixed-Curvature Transformers for Graph Representation Learning [77.1421343649344]
  We propose a generalization of Transformers towards operating entirely on the product of constant curvature spaces. We also provide a kernelized approach to non-Euclidean attention, which enables our model to run in time and memory cost linear to the number of nodes and edges.
  arXiv  Detail & Related papers  (2023-09-08T02:44:37Z)
• Semi-Supervised Manifold Learning with Complexity Decoupled Chart Autoencoders [45.29194877564103]
  This work introduces a chart autoencoder with an asymmetric encoding-decoding process that can incorporate additional semi-supervised information such as class labels. We discuss the approximation power of such networks and derive a bound that essentially depends on the intrinsic dimension of the data manifold rather than the dimension of ambient space.
  arXiv  Detail & Related papers  (2022-08-22T19:58:03Z)
• Learning Flat Latent Manifolds with VAEs [16.725880610265378]
  We propose an extension to the framework of variational auto-encoders, where the Euclidean metric is a proxy for the similarity between data points. We replace the compact prior typically used in variational auto-encoders with a recently presented, more expressive hierarchical one. We evaluate our method on a range of data-sets, including a video-tracking benchmark.
  arXiv  Detail & Related papers  (2020-02-12T09:54:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.