Geometric--Nongeometric Optimizer Calculus: A Modular Language for Reachable Gradient Methods

• URL: http://arxiv.org/abs/2607.07206v1
• Date: Wed, 08 Jul 2026 09:40:44 GMT
• Title: Geometric--Nongeometric Optimizer Calculus: A Modular Language for Reachable Gradient Methods
• Abstract summary: We introduce geometric--nongeometric calculus, a modular language for auditing reachable gradient methods.<n>The main formal result is a direction-expressivity theorem: away from critical points, full positive-definite geometry expresses exactly the strict descent directions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adaptive optimizers mix several mechanisms: a metric or preconditioner maps gradients to descent directions, while estimation, memory, step-size control, constraints, stochasticity, target modification, and discretization determine which directions are available and how they are used. We introduce geometric--nongeometric optimizer calculus, a modular language for auditing reachable gradient methods under explicit oracle, budget, state, and rule constraints. The geometric module is a positive cometric family that maps covectors to parameter-space directions; the nongeometric modules are information, memory, control, operator, noise, target, and discretization mechanisms. The main formal result is a direction-expressivity theorem: away from critical points, full positive-definite geometry expresses exactly the strict descent directions. We then define restricted direction residuals for admissible metric families, prove exact expressivity conditions for diagonal and block geometries, and separate this direction-level diagnostic from condition-number geometric complexity. The resulting design problem is a Pareto optimization over module budgets, not a single universal optimizer ordering. We also lift pointwise residuals to a trajectory-level residual complexity that couples direction mismatch with the variation of the explaining geometry. We include diagnostic prototypes only as evidence for the language: a high-information full-metric probe solves deterministic quadratic benchmarks to numerical precision, while a practical Muon-style PyTorch candidate gives small-scale evidence that matrix-operator updates can be audited by the calculus. The paper is a theory and benchmark-language manuscript; it does not claim large-scale optimizer state-of-the-art performance.

Related papers

• Learning Bidirectional Causal Interactions with Heteroscedastic Neural Networks [1.0356094515229846]
  This paper proposes SEM-DNN, a heteroscedastic neural simultaneous-equation estimator.<n>It learns reciprocal structural interactions without external instruments.<n>An application to ready-to-eat cereal scanner data illustrates how the method can study contemporaneous price-sales feedback.
  arXiv  Detail & Related papers  (2026-07-24T13:53:58Z)
• Restricted Dynamic Geometric Complexity: Certificates for Structured Preconditioning [0.0]
  This paper records that benchmark in the present notation and develops restricted geometric complexity.<n>Main proved results are monotonicity and submanifold-distance principles, diagonal and block reachability as linear inequality feasibility problems.<n>The repository includes toy and synthetic matrix that check diagonal expression gaps, block primal/dual certificates, Kronecker spectral width, and Hessian-relative Kronecker candidate certificates on small quadratic instances.
  arXiv  Detail & Related papers  (2026-07-08T09:36:45Z)
• Optimization Geometrodynamics: A Framework for Dynamic Geometric Optimization [0.0]
  We introduce optimization geometro, a benchmark language for convex-based optimization.<n>We introduce geometric complexity, the minimum geometric cost required to reduce an optimization difficulty.
  arXiv  Detail & Related papers  (2026-07-07T18:42:00Z)
• Generalized nonparametric regression in reproducing kernel Hilbert spaces: Consistency and rates of convergence [0.0]
  We develop a comprehensive theory for M-lev in reproducing spaces.<n>For Sobo spaces we obtain new rates that connect to spaces of smoothness with mixed smoothness functions.
  arXiv  Detail & Related papers  (2026-06-22T08:12:31Z)
• Hybrid Uncertainty Sensitivity Analysis Based on the HSIC for High-Dimensional Responses with Aleatory--Epistemic Separation [4.666836674804876]
  A novel double-space tensor-product RKHS framework is proposed for sensitivity analysis under hybrid uncertainty.<n>The resulting dimension-wise sensitivity indices preserve the uncertainty attribution structure across all output dimensions.<n> Numerical studies on a modified multi-output Ishigami function and an aerodynamic pressure-field problem demonstrate the accuracy, scalability, and practical applicability of the proposed framework.
  arXiv  Detail & Related papers  (2026-06-12T02:58:28Z)
• Fitting Unknown Number of Hyperplanes with Manifold Optimization [57.48093263119306]
  Fitting unknown number of linearplanes to data is a fundamental yet challenging problem in machine learning.<n>Existing approaches often struggle with optimum optima or lack geometric consistency.
  arXiv  Detail & Related papers  (2026-05-27T14:02:20Z)
• Neural Bayesian Sequential Routing [0.0]
  We introduce textbfNeural Bayesian Sequential Routing (NBSR), a framework that models neural inference as active evidence accumulation over a hierarchical Directed Acyclic Graph (DAG)<n>NBSR achieves competitive predictive performance while providing transparent routing traces, path-dependent evidence attribution, uncertainty-aware decision control, and resource-rational inference.<n>Overall, NBSR offers a mathematically grounded framework for interpretable, modular, and resource-rational agentic AI.
  arXiv  Detail & Related papers  (2026-05-22T22:45:56Z)
• Differentiable SpaTiaL: Symbolic Learning and Reasoning with Geometric Temporal Logic for Manipulation Tasks [10.590774471054255]
  Executing complex manipulation in cluttered environments requires satisfying geometric and temporal constraints.<n>We propose Differentiable SpaTiaL, a fully tensorized toolbox that constructs smooth, autograd-compatible geometric primitives directly over polygonal sets.<n>By deriving differentiable relaxations of key spatial predicates, we enable an end-to-end different semanticiable mapping from high-level specifications to low-level geometric configurations.
  arXiv  Detail & Related papers  (2026-04-03T02:13:47Z)
• Deterministic Zeroth-Order Mirror Descent via Vector Fields with A Posteriori Certification [45.85698554568285]
  We develop a deterministic zeroth-order mirror descent framework by replacing gradients with a general vector field.<n>Our analysis provides an evaluation template for last-rate inequality evaluations.<n>Together, these results expose a hidden geometric linking Bregman identities, deterministic certification, and robust conic geometry in zeroth-order mirror descent.
  arXiv  Detail & Related papers  (2026-01-31T10:05:05Z)
• Learning Geometry: A Framework for Building Adaptive Manifold Models through Metric Optimization [8.201374511929538]
  This paper proposes a novel paradigm for machine learning that moves beyond traditional parameter optimization.<n>We optimize the metric tensor field on a manifold with a predefined topology, thereby dynamically shaping the geometric structure of the model space.<n>This work lays a solid foundation for constructing fully dynamic "meta-learners" capable of autonomously evolving their geometry and topology.
  arXiv  Detail & Related papers  (2025-10-30T01:53:32Z)
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• Pure Exploration via Frank-Wolfe Self-Play [9.025261095806309]
  We study pure exploration in structured multi-armed bandits, aiming to efficiently identify the correct hypothesis from a finite set of alternatives.<n>Our analysis proceeds through a continuous-time steer: a differential inclusion with a Lyapunov function that decays, implying a vanishing duality gap and convergence to the optimal value.
  arXiv  Detail & Related papers  (2025-09-24T08:55:21Z)
• Topolow: Force-Directed Euclidean Embedding of Dissimilarity Data with Robustness Against Non-Metricity and Sparsity [0.8287206589886881]
  Topolow is a physics-inspired, gradient-free optimization framework for such embedding problems.<n>Topolow does not require the input dissimilarities to be metric, making it a robust solution for embedding non-metric measurements into a valid Euclidean space.<n>This paper formalizes the algorithm, first introduced as Topolow in the context of antigenic mapping in (Arhami and Rohani, 2025)
  arXiv  Detail & Related papers  (2025-08-03T12:19:17Z)
• Fully Geometric Multi-Hop Reasoning on Knowledge Graphs with Transitive Relations [50.05281461410368]
  We introduce GeometrE, a geometric embedding method for multi-hop reasoning.<n>It does not require learning the logical operations and enables full geometric interpretability.<n>Our experiments show that GeometrE outperforms current state-of-the-art methods on standard benchmark datasets.
  arXiv  Detail & Related papers  (2025-05-18T11:17:50Z)
• Warped geometric information on the optimisation of Euclidean functions [43.43598316339732]
  We consider optimisation of a real-valued function defined in a potentially high-dimensional Euclidean space. We find the function's optimum along a manifold with a warped metric. Our proposed algorithm, using 3rd-order approximation of geodesics, tends to outperform standard Euclidean gradient-based counterparts.
  arXiv  Detail & Related papers  (2023-08-16T12:08:50Z)
• Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior [51.60683890503293]
  We propose a theoretical framework for studying behavior cloning of complex expert demonstrations using generative modeling. We show that pure supervised cloning can generate trajectories matching the per-time step distribution of arbitrary expert trajectories.
  arXiv  Detail & Related papers  (2023-07-27T04:27:26Z)
• Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression [53.15502562048627]
  Recent work has built the connection between self-supervised learning and the approximation of the top eigenspace of a graph Laplacian operator. This work delves into a statistical analysis of augmentation-based pretraining.
  arXiv  Detail & Related papers  (2023-06-01T15:18:55Z)
• Hybrid Predictive Coding: Inferring, Fast and Slow [62.997667081978825]
  We propose a hybrid predictive coding network that combines both iterative and amortized inference in a principled manner. We demonstrate that our model is inherently sensitive to its uncertainty and adaptively balances balances to obtain accurate beliefs using minimum computational expense.
  arXiv  Detail & Related papers  (2022-04-05T12:52:45Z)
• Neuro-Symbolic Entropy Regularization [78.16196949641079]
  In structured prediction, the goal is to jointly predict many output variables that together encode a structured object. One approach -- entropy regularization -- posits that decision boundaries should lie in low-probability regions. We propose a loss, neuro-symbolic entropy regularization, that encourages the model to confidently predict a valid object.
  arXiv  Detail & Related papers  (2022-01-25T06:23:10Z)
• Geometry-aware Bayesian Optimization in Robotics using Riemannian Mat\'ern Kernels [64.62221198500467]
  We show how to implement geometry-aware kernels for Bayesian optimization. This technique can be used for control parameter tuning, parametric policy adaptation, and structure design in robotics.
  arXiv  Detail & Related papers  (2021-11-02T09:47:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.