On the Convergence of Optimizing Persistent-Homology-Based Losses
- URL: http://arxiv.org/abs/2206.02946v1
- Date: Mon, 6 Jun 2022 23:22:37 GMT
- Title: On the Convergence of Optimizing Persistent-Homology-Based Losses
- Authors: Yikai Zhang, Jiachen Yao, Yusu Wang, Chao Chen
- Abstract summary: A topological loss enforces the model to achieve certain desired topological property.
We introduce a general purpose regularized topology-aware loss.
These contributions lead to a new loss function that not only enforces the model to have desired topological behavior, but also achieves satisfying convergence behavior.
- Score: 16.308134813298867
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Topological loss based on persistent homology has shown promise in various
applications. A topological loss enforces the model to achieve certain desired
topological property. Despite its empirical success, less is known about the
optimization behavior of the loss. In fact, the topological loss involves
combinatorial configurations that may oscillate during optimization. In this
paper, we introduce a general purpose regularized topology-aware loss. We
propose a novel regularization term and also modify existing topological loss.
These contributions lead to a new loss function that not only enforces the
model to have desired topological behavior, but also achieves satisfying
convergence behavior. Our main theoretical result guarantees that the loss can
be optimized efficiently, under mild assumptions.
Related papers
- Diffeomorphic interpolation for efficient persistence-based topological optimization [3.7550827441501844]
Topological Data Analysis (TDA) provides a pipeline to extract quantitative topological descriptors from structured objects.
We show that our approach combines efficiently with subsampling techniques, as the diffeomorphism derived from the gradient computed on a subsample can be used to update the coordinates of the full input object.
We also showcase the relevance of our approach for black-box autoencoder (AE) regularization, where we aim at enforcing topological priors on the latent spaces associated to fixed, pre-trained, blackbox AE models.
arXiv Detail & Related papers (2024-05-29T07:00:28Z) - On the Dynamics Under the Unhinged Loss and Beyond [104.49565602940699]
We introduce the unhinged loss, a concise loss function, that offers more mathematical opportunities to analyze closed-form dynamics.
The unhinged loss allows for considering more practical techniques, such as time-vary learning rates and feature normalization.
arXiv Detail & Related papers (2023-12-13T02:11:07Z) - A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime.
We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
arXiv Detail & Related papers (2023-11-13T01:48:08Z) - Implicit Bias of Gradient Descent for Logistic Regression at the Edge of
Stability [69.01076284478151]
In machine learning optimization, gradient descent (GD) often operates at the edge of stability (EoS)
This paper studies the convergence and implicit bias of constant-stepsize GD for logistic regression on linearly separable data in the EoS regime.
arXiv Detail & Related papers (2023-05-19T16:24:47Z) - Gradient descent provably escapes saddle points in the training of
shallow ReLU networks [3.0079490585515343]
We prove a variant of the relevant dynamical systems result, a center-stable manifold theorem, in which we relax some regularity requirements.
Building on a classification of critical points of the square integral loss of shallow ReLU networks measured against an affine target function, we deduce that gradient descent avoids most saddle points.
arXiv Detail & Related papers (2022-08-03T14:08:52Z) - Topologically penalized regression on manifolds [0.0]
We study a regression problem on a compact manifold M.
In order to take advantage of the underlying geometry and topology of the data, the regression task is performed on the basis of the first several eigenfunctions of the Laplace-Beltrami operator of the manifold.
The proposed penalties are based on the topology of the sub-level sets of either the eigenfunctions or the estimated function.
arXiv Detail & Related papers (2021-10-26T14:59:13Z) - Fractal Structure and Generalization Properties of Stochastic
Optimization Algorithms [71.62575565990502]
We prove that the generalization error of an optimization algorithm can be bounded on the complexity' of the fractal structure that underlies its generalization measure.
We further specialize our results to specific problems (e.g., linear/logistic regression, one hidden/layered neural networks) and algorithms.
arXiv Detail & Related papers (2021-06-09T08:05:36Z) - Risk Guarantees for End-to-End Prediction and Optimization Processes [0.0]
We study conditions that allow us to explicitly describe how the prediction performance governs the optimization performance.
We derive the exact theoretical relationship between prediction performance measured with the squared loss, as well as a class of symmetric loss functions, and the subsequent optimization performance.
arXiv Detail & Related papers (2020-12-30T05:20:26Z) - Efficient Methods for Structured Nonconvex-Nonconcave Min-Max
Optimization [98.0595480384208]
We propose a generalization extraient spaces which converges to a stationary point.
The algorithm applies not only to general $p$-normed spaces, but also to general $p$-dimensional vector spaces.
arXiv Detail & Related papers (2020-10-31T21:35:42Z) - On dissipative symplectic integration with applications to
gradient-based optimization [77.34726150561087]
We propose a geometric framework in which discretizations can be realized systematically.
We show that a generalization of symplectic to nonconservative and in particular dissipative Hamiltonian systems is able to preserve rates of convergence up to a controlled error.
arXiv Detail & Related papers (2020-04-15T00:36:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.