LAST: Bridging Vision-Language and Action Manifolds via Gromov-Wasserstein Alignment

• URL: http://arxiv.org/abs/2606.11221v1
• Date: Wed, 27 May 2026 03:48:10 GMT
• Title: LAST: Bridging Vision-Language and Action Manifolds via Gromov-Wasserstein Alignment
• Abstract summary: LAST reconstructs the action space to establish local metric compatibility with the Vision-Language-Action modality.<n> LAST enables VLA models with superior convergence and generalizability.
• Score: 71.3712075841632
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We take a Gromov-Wasserstein perspective on Vision-Language-Action (VLA) learning, where the goal is to make the relational geometry of action representations compatible with the semantic geometry of VL embeddings. However, this alignment is non-trivial due to the mathematical heterogeneity between the domains: the semantic space of vision-language is topologically linear and isotropic, whereas the physical manifold of robotic action is non-Euclidean and anisotropic. Their disjoint metric structures render direct regression ill-posed. To resolve this incompatibility, we introduce LAST (Lie-algebraic Action Space Tokenizer), which reconstructs the action space to establish local metric compatibility with the VL modality via a two-stage transformation: (1) Global Topological Linearization: linearizing the action manifold via Lie-algebraic mapping, converting trajectories into a fixed-length, physically additive representation. (2) Local Metric Discretization: hierarchically discretizing the representation into schemas and whitened residuals, yielding approximately isotropic local charts that are statistically aligned with the semantic metric. By resolving the structural mismatch at both global and local levels, LAST enables VLA models with superior convergence and generalizability.

Related papers

• General Covariant Action Modeling: Constructing Generalized Manifolds via Spatio-Temporal Decoupling [48.559681359374146]
  Generalized Action Manifold (GAM) framework enforces general covariance through structural disentanglement.<n>GAM enables superior transfer and robustness capabilities, outperforming geometry-agnostic baselines.
  arXiv  Detail & Related papers  (2026-05-27T03:38:15Z)
• Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers [93.3976834364707]
  We propose sREPA, a structural REPresentation Alignment framework to enforce consistency in the relational geometry of feature maps.<n>By encouraging the model to internalize holistic spatial layouts and structural correlations from pre-trained features, sREPA achieves faster and more stable convergence.
  arXiv  Detail & Related papers  (2026-05-16T12:01:04Z)
• Hyperbolic Enhanced Representation Learning for Incomplete Multi-view Clustering [57.38215918201251]
  We propose HERL, a Hyperbolic Enhanced Representation Learning framework for Incomplete Multi-View Clustering.<n> operating within the Poincaré ball, HERL constructs a structure-aware latent space to enhance representation learning.<n>We show that HERL consistently outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2026-04-18T10:50:46Z)
• INST-Align: Implicit Neural Alignment for Spatial Transcriptomics via Canonical Expression Fields [1.0487944945684993]
  INST-Align couples a coordinate-based deformation network with a shared Canonical Expression Field.<n>It achieves state-of-the-art mean OT Accuracy (0.702), NN Accuracy (0.719), and Chamfer distance, with Chamfer reductions of up to 94.9% on large-deformation sections.<n>It also yields biologically meaningful spatial embeddings and coherent 3D tissue reconstruction.
  arXiv  Detail & Related papers  (2026-04-13T21:44:18Z)
• LAG-XAI: A Lie-Inspired Affine Geometric Framework for Interpretable Paraphrasing in Transformer Latent Spaces [1.3788139387418392]
  This paper introduces LAG-XAI, a geometric framework that models paraphrasing as a structured affine transformation within the embedding space.<n>Experiments on the noisy PIT-2015 Twitter corpus, encoded with Sentence-BERT, reveal a "linear transparency" phenomenon.<n>The model captures approximately 80% of the non-linear baseline's effective classification capacity (AUC 0.8405), offering explicit interpretability in exchange for a marginal drop in absolute accuracy.
  arXiv  Detail & Related papers  (2026-04-07T17:02:21Z)
• From Directions to Regions: Decomposing Activations in Language Models via Local Geometry [37.50120706345745]
  We leverage Mixture of Factor Analyzers (MFA) as a scalable, unsupervised alternative that models the activation space.<n>MFA decomposes activations into two compositional geometric objects: the region's centroid in activation space, and the local variation from the centroid.<n>We train large-scale MFAs for Llama-3.1-8B and Gemma-2-2B, and show they capture complex, nonlinear structures in activation space.
  arXiv  Detail & Related papers  (2026-02-02T18:49:05Z)
• Relative Representations: Topological and Geometric Perspectives [50.85040046976025]
  Relative representations are an established approach to zero-shot model stitching.<n>We introduce a normalization procedure in the relative transformation, resulting in invariance to non-isotropic rescalings and permutations.<n>Second, we propose to deploy topological densification when fine-tuning relative representations, a topological regularization loss encouraging clustering within classes.
  arXiv  Detail & Related papers  (2024-09-17T08:09:22Z)
• HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learning [74.76431541169342]
  Zero-shot learning (ZSL) tackles the unseen class recognition problem, transferring semantic knowledge from seen classes to unseen ones. We propose a novel hierarchical semantic-visual adaptation (HSVA) framework to align semantic and visual domains. Experiments on four benchmark datasets demonstrate HSVA achieves superior performance on both conventional and generalized ZSL.
  arXiv  Detail & Related papers  (2021-09-30T14:27:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.