Stochastic Langevin Differential Inclusions with Applications to Machine Learning

• URL: http://arxiv.org/abs/2206.11533v3
• Date: Sun, 12 May 2024 14:32:43 GMT
• Title: Stochastic Langevin Differential Inclusions with Applications to Machine Learning
• Authors: Fabio V. Difonzo, Vyacheslav Kungurtsev, Jakub Marecek,
• Abstract summary: We show some foundational results regarding the flow and properties of Langevin-type Differential Inclusions. In particular, we show strong existence of the solution, as well as an canonical- minimization of the free-energy functional.
• Score: 5.274477003588407
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Stochastic differential equations of Langevin-diffusion form have received significant attention, thanks to their foundational role in both Bayesian sampling algorithms and optimization in machine learning. In the latter, they serve as a conceptual model of the stochastic gradient flow in training over-parameterized models. However, the literature typically assumes smoothness of the potential, whose gradient is the drift term. Nevertheless, there are many problems for which the potential function is not continuously differentiable, and hence the drift is not Lipschitz continuous everywhere. This is exemplified by robust losses and Rectified Linear Units in regression problems. In this paper, we show some foundational results regarding the flow and asymptotic properties of Langevin-type Stochastic Differential Inclusions under assumptions appropriate to the machine-learning settings. In particular, we show strong existence of the solution, as well as an asymptotic minimization of the canonical free-energy functional.

Related papers

Err

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.