Diffeomorphic Optimization

• URL: http://arxiv.org/abs/2607.00947v1
• Date: Wed, 01 Jul 2026 13:46:22 GMT
• Title: Diffeomorphic Optimization
• Abstract summary: Generative models learn data distributions on a low-dimensional manifold within a higher-dimensional ambient space.<n>We show this is equivalent to gradient descent on the data up to $mathcalO(2)$ corrections, keeping trajectories on-manifold by construction.
• Score: 7.066746852538784
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative models learn data distributions that reside on a low-dimensional manifold within a higher-dimensional ambient space. Optimizing differentiable objectives on this manifold is challenging: the ambient loss landscape is high-dimensional, rugged, and non-convex. Direct gradient descent, blind to the manifold's geometry, quickly drifts off it. Diffeomorphic optimization starts from the observation that diffusion and flow models provide a map from the data manifold to a much simpler base space in which we perform gradient descent. Using differential geometry, we show this is equivalent to Riemannian gradient descent on the data manifold up to $\mathcal{O}(λ^2)$ corrections, keeping trajectories on-manifold by construction and yielding a smoother optimization surface. For protein design, we extend diffeomorphic optimization to the matrix Lie groups $\mathrm{SO}(3)$ and $\mathrm{SE}(3)$, deriving an autograd-compatible $\mathrm{SO}(3)$ gradient and a generalized adjoint-state method for backpropagation through Lie-group ODE solvers. Diffeomorphic optimization improves over tuned guidance on secondary-structure targeting with FrameFlow ($91.3\%$ vs. $63.3\%$ of residues in the Ramachandran target), outperforms OC-Flow on peptide binding affinity at $2\times$ the speed, and reduces Rosetta energies by thousands of units across the PDB test set for structures with hundreds of residues.

Related papers

• FlatVPR: Plug-and-play Geo-linear Residual Adapter for Geometric Rectification of Foundation Model Feature Manifolds [0.7734726150561086]
  FlatVPR'' bridges the trade-off between map lightweightness and localization accuracy in visual place recognition.<n>Our method explicitly suppresses manifold curvature using a mathematically grounded Pullback Flatness Loss.<n>Experiments on the NCLT dataset demonstrate that the application of our adapter leads to significant performance improvements.
  arXiv  Detail & Related papers  (2026-06-01T05:56:59Z)
• Intrinsic Wasserstein Rates for Score-Based Generative Models on Smooth Manifolds [61.14405512940818]
  We show that Score-based generative models are trained in high-dimensional ambient spaces.<n>We use a ReLU implementation of nearest-projection coordinates via finite intrinsic anchors and Gauss Newton.
  arXiv  Detail & Related papers  (2026-05-15T10:20:05Z)
• Towards Scalable Persistence-Based Topological Optimization [44.16669776030478]
  Persistence-based topological optimization deforms a point cloud $X subset mathbbRd$ by minimizing objectives of the form $L(X) = ell(mathrmDgm(X))$, where $mathrmDgm(X)$ is a persistence diagram.<n>In practice, optimization is limited by two coupled issues: persistent homology is typically computed on subsamples, and the resulting topological gradients are highly sparse, with only a few anchor points receiving nonzero updates.
  arXiv  Detail & Related papers  (2026-05-09T15:47:20Z)
• StelLA: Subspace Learning in Low-rank Adaptation using Stiefel Manifold [51.93627542334909]
  Low-rank adaptation (LoRA) has been widely adopted as a parameter-efficient technique for fine-tuning large-scale pre-trained models.<n>We propose a geometry-aware extension of LoRA that uses a three-factor decomposition $U!SVtop$.
  arXiv  Detail & Related papers  (2025-10-02T11:59:13Z)
• Efficient Diffusion Models for Symmetric Manifolds [25.99200001269046]
  We introduce a framework for designing efficient diffusion models for $d$-dimensional symmetric-space.<n>Mandela symmetries ensure the diffusion satisfies an "average-case" Lipschitz condition.<n>Our model outperforms prior methods in training speed and improves sample quality on synthetic datasets.
  arXiv  Detail & Related papers  (2025-05-27T18:12:29Z)
• FFT-based Dynamic Subspace Selection for Low-Rank Adaptive Optimization of Large Language Models [49.397861654088636]
  We propose a two-step procedure to approximate SVD/QR-based gradient projections into lower-dimensional spaces.<n>We show that our strategy achieves faster runtime and reduced memory usage by up to $25%$ across different model sizes.
  arXiv  Detail & Related papers  (2025-05-23T14:37:00Z)
• 3DGS-LM: Faster Gaussian-Splatting Optimization with Levenberg-Marquardt [65.25603275491544]
  We present 3DGS-LM, a new method that accelerates the reconstruction of 3D Gaussian Splatting (3DGS)<n>Our method is 20% faster than the original 3DGS while obtaining the same reconstruction quality optimization.
  arXiv  Detail & Related papers  (2024-09-19T16:31:44Z)
• Global $\mathcal{L}^2$ minimization at uniform exponential rate via geometrically adapted gradient descent in Deep Learning [1.4050802766699084]
  We consider the scenario of supervised learning in Deep Learning (DL) networks.<n>We choose the gradient flow with respect to the Euclidean metric in the output layer of the DL network.
  arXiv  Detail & Related papers  (2023-11-27T02:12:02Z)
• Scaling Riemannian Diffusion Models [68.52820280448991]
  We show that our method enables us to scale to high dimensional tasks on nontrivial manifold. We model QCD densities on $SU(n)$ lattices and contrastively learned embeddings on high dimensional hyperspheres.
  arXiv  Detail & Related papers  (2023-10-30T21:27:53Z)
• ELRA: Exponential learning rate adaption gradient descent optimization method [83.88591755871734]
  We present a novel, fast (exponential rate), ab initio (hyper-free) gradient based adaption. The main idea of the method is to adapt the $alpha by situational awareness. It can be applied to problems of any dimensions n and scales only linearly.
  arXiv  Detail & Related papers  (2023-09-12T14:36:13Z)
• IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation [44.00115413716392]
  This paper presents an inverse kinematic layer (IKOL) for 3D human pose shape estimation. IKOL has a much over over than most existing regression-based methods. It provides a more accurate range of 3D human pose estimation.
  arXiv  Detail & Related papers  (2023-02-02T12:43:29Z)
• Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to Optimization [50.83356836818667]
  gradient Langevin Dynamics is one of the most fundamental algorithms to solve non-eps optimization problems. In this paper, we show two variants of this kind, namely the Variance Reduced Langevin Dynamics and the Recursive Gradient Langevin Dynamics.
  arXiv  Detail & Related papers  (2022-03-30T11:39:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.