Delaunay-Triangulation-Based Learning with Hessian Total-Variation Regularization

• URL: http://arxiv.org/abs/2208.07787v1
• Date: Tue, 16 Aug 2022 14:55:05 GMT
• Title: Delaunay-Triangulation-Based Learning with Hessian Total-Variation Regularization
• Abstract summary: In this paper, we propose an alternative method that leverages the expressivity of CPWL functions. Our approach relies on the partitioning of the domain of the CPWL function by a Delaunay triangulation. Our experiments validate the usage of our method in low-dimensional scenarios.
• Score: 21.69222364939501
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Regression is one of the core problems tackled in supervised learning. Rectified linear unit (ReLU) neural networks generate continuous and piecewise-linear (CPWL) mappings and are the state-of-the-art approach for solving regression problems. In this paper, we propose an alternative method that leverages the expressivity of CPWL functions. In contrast to deep neural networks, our CPWL parameterization guarantees stability and is interpretable. Our approach relies on the partitioning of the domain of the CPWL function by a Delaunay triangulation. The function values at the vertices of the triangulation are our learnable parameters and identify the CPWL function uniquely. Formulating the learning scheme as a variational problem, we use the Hessian total variation (HTV) as regularizer to favor CPWL functions with few affine pieces. In this way, we control the complexity of our model through a single hyperparameter. By developing a computational framework to compute the HTV of any CPWL function parameterized by a triangulation, we discretize the learning problem as the generalized least absolute shrinkage and selection operator (LASSO). Our experiments validate the usage of our method in low-dimensional scenarios.

Related papers

• Three-dimensional narrow volume reconstruction method with unconditional stability based on a phase-field Lagrange multiplier approach [3.0012646624584245]
  Reconstruction of an object from points cloud is essential in prosthetics, medical imaging, computer vision, etc.<n>We present an effective algorithm for an Allen--Cahn-type model of reconstruction, employing the Lagrange multiplier approach.<n> Comprehensive numerical experiments, including reconstructions of complex 3D volumes such as characters from textitStar Wars, validate the algorithm's accuracy, stability, and effectiveness.
  arXiv  Detail & Related papers  (2025-11-01T11:21:12Z)
• Deep Equilibrium models for Poisson Imaging Inverse problems via Mirror Descent [7.248102801711294]
  Deep Equilibrium Models (DEQs) are implicit neural networks with fixed points.<n>We introduce a novel DEQ formulation based on Mirror Descent defined in terms of a tailored non-Euclidean geometry.<n>We propose computational strategies that enable both efficient training and fully parameter-free inference.
  arXiv  Detail & Related papers  (2025-07-15T16:33:01Z)
• Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum [78.27945336558987]
  Decentralized server (DFL) eliminates reliance on client-client architecture. Non-smooth regularization is often incorporated into machine learning tasks. We propose a novel novel DNCFL algorithm to solve these problems.
  arXiv  Detail & Related papers  (2025-04-17T08:32:25Z)
• Weak instrumental variables due to nonlinearities in panel data: A Super Learner Control Function estimator [0.0]
  We propose a triangular simultaneous equation model for panel data with additive separable individual-specific fixed effects. The estimation procedure is composed of two main steps and sample splitting. We conclude that the Super Learner Control Function Estimators significantly outperform Within 2SLS estimators.
  arXiv  Detail & Related papers  (2025-04-04T07:22:18Z)
• Kolmogorov-Arnold Transformer [72.88137795439407]
  We introduce the Kolmogorov-Arnold Transformer (KAT), a novel architecture that replaces layers with Kolmogorov-Arnold Network (KAN) layers. We identify three key challenges: (C1) Base function, (C2) Inefficiency, and (C3) Weight. With these designs, KAT outperforms traditional-based transformers.
  arXiv  Detail & Related papers  (2024-09-16T17:54:51Z)
• A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation [66.26739783789387]
  We propose a new algorithm, Monotonic Q-Learning with Upper Confidence Bound (MQL-UCB) for reinforcement learning. MQL-UCB achieves minimax optimal regret of $tildeO(dsqrtHK)$ when $K$ is sufficiently large and near-optimal policy switching cost. Our work sheds light on designing provably sample-efficient and deployment-efficient Q-learning with nonlinear function approximation.
  arXiv  Detail & Related papers  (2023-11-26T08:31:57Z)
• Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEs [93.82811501035569]
  We introduce a new data efficient and highly parallelizable operator learning approach with reduced memory requirement and better generalization. MG-TFNO scales to large resolutions by leveraging local and global structures of full-scale, real-world phenomena. We demonstrate superior performance on the turbulent Navier-Stokes equations where we achieve less than half the error with over 150x compression.
  arXiv  Detail & Related papers  (2023-09-29T20:18:52Z)
• Message Passing Neural PDE Solvers [60.77761603258397]
  We build a neural message passing solver, replacing allally designed components in the graph with backprop-optimized neural function approximators. We show that neural message passing solvers representationally contain some classical methods, such as finite differences, finite volumes, and WENO schemes. We validate our method on various fluid-like flow problems, demonstrating fast, stable, and accurate performance across different domain topologies, equation parameters, discretizations, etc., in 1D and 2D.
  arXiv  Detail & Related papers  (2022-02-07T17:47:46Z)
• Sparsest Univariate Learning Models Under Lipschitz Constraint [31.28451181040038]
  We propose continuous-domain formulations for one-dimensional regression problems. We control the Lipschitz constant explicitly using a user-defined upper-bound. We show that both problems admit global minimizers that are continuous and piecewise-linear.
  arXiv  Detail & Related papers  (2021-12-27T07:03:43Z)
• A Discontinuity Capturing Shallow Neural Network for Elliptic Interface Problems [0.0]
  Discontinuity Capturing Shallow Neural Network (DCSNN) for approximating $d$-dimensional piecewise continuous functions and for solving elliptic interface problems is developed. DCSNN model is comparably efficient due to only moderate number of parameters needed to be trained.
  arXiv  Detail & Related papers  (2021-06-10T08:40:30Z)
• Ridge regression with adaptive additive rectangles and other piecewise functional templates [0.0]
  We propose an $L_2$-based penalization algorithm for functional linear regression models. We show how our algorithm alternates between approximating a suitable template and solving a convex ridge-like problem.
  arXiv  Detail & Related papers  (2020-11-02T15:28:54Z)
• Logistic Q-Learning [87.00813469969167]
  We propose a new reinforcement learning algorithm derived from a regularized linear-programming formulation of optimal control in MDPs. The main feature of our algorithm is a convex loss function for policy evaluation that serves as a theoretically sound alternative to the widely used squared Bellman error.
  arXiv  Detail & Related papers  (2020-10-21T17:14:31Z)
• Neural Control Variates [71.42768823631918]
  We show that a set of neural networks can face the challenge of finding a good approximation of the integrand. We derive a theoretically optimal, variance-minimizing loss function, and propose an alternative, composite loss for stable online training in practice. Specifically, we show that the learned light-field approximation is of sufficient quality for high-order bounces, allowing us to omit the error correction and thereby dramatically reduce the noise at the cost of negligible visible bias.
  arXiv  Detail & Related papers  (2020-06-02T11:17:55Z)
• Finite-Sample Analysis of Stochastic Approximation Using Smooth Convex Envelopes [40.31139355952393]
  We construct a smooth Lyapunov function using the generalized envelope, and show that the iterates of SA have negative drift with respect to that Lyapunov function. In particular, we use it to establish the first-known convergence rate of the V-trace algorithm for off-policy TD-learning. We also use it to study TD-learning in the on-policy setting, and recover the existing state-of-the-art results for $Q$-learning.
  arXiv  Detail & Related papers  (2020-02-03T16:42:01Z)
• Deep Neural Networks with Trainable Activations and Controlled Lipschitz Constant [26.22495169129119]
  We introduce a variational framework to learn the activation functions of deep neural networks. Our aim is to increase the capacity of the network while controlling an upper-bound of the Lipschitz constant. We numerically compare our scheme with standard ReLU network and its variations, PReLU and LeakyReLU.
  arXiv  Detail & Related papers  (2020-01-17T12:32:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.