A kinetic-based regularization method for data science applications

• URL: http://arxiv.org/abs/2503.04857v1
• Date: Thu, 06 Mar 2025 08:12:01 GMT
• Title: A kinetic-based regularization method for data science applications
• Authors: Abhisek Ganguly, Alessandro Gabbana, Vybhav Rao, Sauro Succi, Santosh Ansumali,
• Abstract summary: We propose a physics-based regularization technique for function learning, inspired by statistical mechanics.<n>By drawing an analogy between optimizing the parameters of an interpolator and minimizing the energy of a system, we introduce constraints on the lower-order moments of the data distribution.<n>This minimizes the discrepancy between the discrete and continuum representations of the data, in turn allowing to access more favorable energy landscapes.
• Score: 39.58317527488534
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a physics-based regularization technique for function learning, inspired by statistical mechanics. By drawing an analogy between optimizing the parameters of an interpolator and minimizing the energy of a system, we introduce corrections that impose constraints on the lower-order moments of the data distribution. This minimizes the discrepancy between the discrete and continuum representations of the data, in turn allowing to access more favorable energy landscapes, thus improving the accuracy of the interpolator. Our approach improves performance in both interpolation and regression tasks, even in high-dimensional spaces. Unlike traditional methods, it does not require empirical parameter tuning, making it particularly effective for handling noisy data. We also show that thanks to its local nature, the method offers computational and memory efficiency advantages over Radial Basis Function interpolators, especially for large datasets.

Related papers

• Generalized Tensor-based Parameter-Efficient Fine-Tuning via Lie Group Transformations [50.010924231754856]
  Adapting pre-trained foundation models for diverse downstream tasks is a core practice in artificial intelligence. To overcome this, parameter-efficient fine-tuning (PEFT) methods like LoRA have emerged and are becoming a growing research focus. We propose a generalization that extends matrix-based PEFT methods to higher-dimensional parameter spaces without compromising their structural properties.
  arXiv  Detail & Related papers  (2025-04-01T14:36:45Z)
• Preconditioned Additive Gaussian Processes with Fourier Acceleration [2.292881746604941]
  We introduce a matrix-free method to achieve nearly linear complexity in the multiplication of kernel matrices and their derivatives. To address high-dimensional problems, we propose an additive kernel approach. Each sub- Kernel captures lower-order feature interactions, allowing for the efficient application of the NFFT method.
  arXiv  Detail & Related papers  (2025-04-01T07:14:06Z)
• Optimal Transport-Based Displacement Interpolation with Data Augmentation for Reduced Order Modeling of Nonlinear Dynamical Systems [0.0]
  We present a novel reduced-order Model (ROM) that exploits optimal transport theory and displacement to enhance the representation of nonlinear dynamics in complex systems. We show improved accuracy and efficiency in predicting complex system behaviors, indicating the potential of this approach for a wide range of applications in computational physics and engineering.
  arXiv  Detail & Related papers  (2024-11-13T16:29:33Z)
• Efficient Trajectory Inference in Wasserstein Space Using Consecutive Averaging [3.8623569699070353]
  Trajectory inference deals with the challenge of reconstructing continuous processes from such observations. We propose methods for B-spline approximation of point clouds through consecutive averaging that is instrinsic to the Wasserstein space. We rigorously evaluate our method by providing convergence guarantees and testing it on simulated cell data.
  arXiv  Detail & Related papers  (2024-05-30T04:19:20Z)
• Adaptive debiased SGD in high-dimensional GLMs with streaming data [4.704144189806667]
  This paper introduces a novel approach to online inference in high-dimensional generalized linear models.<n>Our method operates in a single-pass mode, making it different from existing methods that require full dataset access or large-dimensional summary statistics storage.<n>The core of our methodological innovation lies in an adaptive descent algorithm tailored for dynamic objective functions, coupled with a novel online debiasing procedure.
  arXiv  Detail & Related papers  (2024-05-28T15:36:48Z)
• Instance-Specific Asymmetric Sensitivity in Differential Privacy [2.855485723554975]
  We build upon previous work that gives a paradigm for selecting an output through the exponential mechanism. Our framework will slightly modify the closeness metric and instead give a simple and efficient application of the sparse vector technique.
  arXiv  Detail & Related papers  (2023-11-02T05:01:45Z)
• Dynamic Kernel-Based Adaptive Spatial Aggregation for Learned Image Compression [63.56922682378755]
  We focus on extending spatial aggregation capability and propose a dynamic kernel-based transform coding. The proposed adaptive aggregation generates kernel offsets to capture valid information in the content-conditioned range to help transform. Experimental results demonstrate that our method achieves superior rate-distortion performance on three benchmarks compared to the state-of-the-art learning-based methods.
  arXiv  Detail & Related papers  (2023-08-17T01:34:51Z)
• FaDIn: Fast Discretized Inference for Hawkes Processes with General Parametric Kernels [82.53569355337586]
  This work offers an efficient solution to temporal point processes inference using general parametric kernels with finite support. The method's effectiveness is evaluated by modeling the occurrence of stimuli-induced patterns from brain signals recorded with magnetoencephalography (MEG) Results show that the proposed approach leads to an improved estimation of pattern latency than the state-of-the-art.
  arXiv  Detail & Related papers  (2022-10-10T12:35:02Z)
• Data-heterogeneity-aware Mixing for Decentralized Learning [63.83913592085953]
  We characterize the dependence of convergence on the relationship between the mixing weights of the graph and the data heterogeneity across nodes. We propose a metric that quantifies the ability of a graph to mix the current gradients. Motivated by our analysis, we propose an approach that periodically and efficiently optimize the metric.
  arXiv  Detail & Related papers  (2022-04-13T15:54:35Z)
• ParK: Sound and Efficient Kernel Ridge Regression by Feature Space Partitions [34.576469570537995]
  We introduce ParK, a new large-scale solver for kernel ridge regression. Our approach combines partitioning with random projections and iterative optimization to reduce space and time complexity.
  arXiv  Detail & Related papers  (2021-06-23T08:24:36Z)
• Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations [79.71184760864507]
  This article introduces a new physics-based method for rigid point set alignment called Fast Gravitational Approach (FGA) In FGA, the source and target point sets are interpreted as rigid particle swarms with masses interacting in a globally multiply-linked manner while moving in a simulated gravitational force field. We show that the new method class has characteristics not found in previous alignment methods.
  arXiv  Detail & Related papers  (2020-09-28T15:05:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.