Hessian-vector products for tensor networks via recursive tangent-state propagation

• URL: http://arxiv.org/abs/2604.20384v1
• Date: Wed, 22 Apr 2026 09:38:08 GMT
• Title: Hessian-vector products for tensor networks via recursive tangent-state propagation
• Abstract summary: We introduce an analytical Hessian-vector product kernel designed for arbitrary compositions of linear maps.<n>Our second-order approach achieves up to a four-order-of-magnitude improvement in fidelity over naive Trotterization.
• Score: 1.5282767384702272
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optimizing tensor networks with standard first-order methods often leads to slow convergence and entrapment in local minima. Although second-order optimization offers enhanced robustness, explicitly constructing the full Hessian matrix is computationally prohibitive for large-scale systems. In this work, we bypass this bottleneck by introducing an analytical Hessian-vector product kernel designed for arbitrary compositions of linear maps. This two-pass algorithm leverages recursive tangent-state propagation with a bounded virtual bond dimension to guarantee scalability. We demonstrate the practical utility of this kernel by integrating it into a Riemannian trust-region framework for quantum circuit compression. Evaluated on time-evolution circuits for various spin chains, our second-order approach achieves up to a four-order-of-magnitude improvement in fidelity over naive Trotterization, while delivering significantly smoother, faster convergence than conventional first-order methods such as Riemannian ADAM.

Related papers

• Unrolled Neural Networks for Constrained Optimization [83.29547301151177]
  Our framework comprises two coupled neural networks that jointly approximate the saddle point of the Lagrangian.<n>We numerically evaluate the framework on mixed-integer quadratic programs and power allocation in wireless networks.
  arXiv  Detail & Related papers  (2026-01-24T03:12:41Z)
• Generalized Gradient Norm Clipping & Non-Euclidean $(L_0,L_1)$-Smoothness [51.302674884611335]
  This work introduces a hybrid non-Euclidean optimization method which generalizes norm clipping by combining steepest descent and conditional gradient approaches.<n>We discuss how to instantiate the algorithms for deep learning and demonstrate their properties on image classification and language modeling.
  arXiv  Detail & Related papers  (2025-06-02T17:34:29Z)
• Distributed Optimization via Energy Conservation Laws in Dilated Coordinates [5.35599092568615]
  This paper introduces an energy conservation approach for analyzing continuous-time dynamical systems in dilated coordinates. convergence rates can be explicitly expressed in terms of the inverse time-dilation factor. Its accelerated convergence behavior is benchmarked against various state-of-the-art distributed optimization algorithms on practical, large-scale problems.
  arXiv  Detail & Related papers  (2024-09-28T08:02:43Z)
• A Riemannian Approach to the Lindbladian Dynamics of a Locally Purified Tensor Network [0.0]
  We propose a framework for implementing Lindbladian dynamics in many-body open quantum systems with nearest-neighbor couplings.<n>In this work, we leverage the gauge freedom inherent in the Kraus representation of quantum channels to improve the splitting error.<n>We validate our approach using two nearest-neighbor noise models and achieve an improvement of orders of magnitude compared to other positivity-preserving schemes.
  arXiv  Detail & Related papers  (2024-09-12T15:16:15Z)
• Fast Dual Subgradient Optimization of the Integrated Transportation Distance Between Stochastic Kernels [1.5229257192293204]
  A generalization of the Wasserstein metric, the integrated transportation distance, establishes a novel distance between probability kernels of Markov systems. This metric serves as the foundation for an efficient approximation technique, enabling the replacement of the original system's kernel with a kernel with a discrete support of limited cardinality. We present a specialized dual algorithm capable of constructing these approximate kernels quickly and efficiently, without requiring computationally expensive matrix operations.
  arXiv  Detail & Related papers  (2023-12-03T15:44:17Z)
• Information-Theoretic Trust Regions for Stochastic Gradient-Based Optimization [17.79206971486723]
  We show that arTuRO combines the fast convergence of adaptive moment-based optimization with the capabilities of SGD. We approximate the diagonal elements of the Hessian from gradients, constructing a model of the expected Hessian over time using only first-order information. We show that arTuRO combines the fast convergence of adaptive moment-based optimization with the capabilities of SGD.
  arXiv  Detail & Related papers  (2023-10-31T16:08:38Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Zeroth-order Riemannian Averaging Stochastic Approximation Algorithms [19.99781875916751]
  We show that textttZo-RASA achieves optimal sample complexities for generating $epsilon$-approximation first-order stationary solutions. We improve the algorithm's practicality by using geometrics and vector transport, instead of exponential mappings and parallel transports.
  arXiv  Detail & Related papers  (2023-09-25T20:13:36Z)
• Fast Computation of Optimal Transport via Entropy-Regularized Extragradient Methods [75.34939761152587]
  Efficient computation of the optimal transport distance between two distributions serves as an algorithm that empowers various applications. This paper develops a scalable first-order optimization-based method that computes optimal transport to within $varepsilon$ additive accuracy.
  arXiv  Detail & Related papers  (2023-01-30T15:46:39Z)
• A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks [56.084798078072396]
  We take a step towards closing the gap between theory and practice by significantly improving the known theoretical bounds on both the network width and the convergence time. We show that convergence to a global minimum is guaranteed for networks with quadratic widths in the sample size and linear in their depth at a time logarithmic in both. Our analysis and convergence bounds are derived via the construction of a surrogate network with fixed activation patterns that can be transformed at any time to an equivalent ReLU network of a reasonable size.
  arXiv  Detail & Related papers  (2021-01-12T00:40:45Z)
• Controllable Orthogonalization in Training DNNs [96.1365404059924]
  Orthogonality is widely used for training deep neural networks (DNNs) due to its ability to maintain all singular values of the Jacobian close to 1. This paper proposes a computationally efficient and numerically stable orthogonalization method using Newton's iteration (ONI) We show that our method improves the performance of image classification networks by effectively controlling the orthogonality to provide an optimal tradeoff between optimization benefits and representational capacity reduction. We also show that ONI stabilizes the training of generative adversarial networks (GANs) by maintaining the Lipschitz continuity of a network, similar to spectral normalization (
  arXiv  Detail & Related papers  (2020-04-02T10:14:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.