Radial-VCReg: More Informative Representation Learning Through Radial Gaussianization

• URL: http://arxiv.org/abs/2602.14272v1
• Date: Sun, 15 Feb 2026 18:50:52 GMT
• Title: Radial-VCReg: More Informative Representation Learning Through Radial Gaussianization
• Authors: Yilun Kuang, Yash Dagade, Deep Chakraborty, Erik Learned-Miller, Randall Balestriero, Tim G. J. Rudner, Yann LeCun,
• Abstract summary: We propose Radial-VCReg, which augments VCReg with a radial Gaussianization loss that aligns feature norms with the Chi distribution.<n>We prove that Radial-VCReg transforms a broader class towards normality compared to VCReg and show on synthetic and real-world datasets that it consistently improves performance.
• Score: 49.236368760189464
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervised learning aims to learn maximally informative representations, but explicit information maximization is hindered by the curse of dimensionality. Existing methods like VCReg address this by regularizing first and second-order feature statistics, which cannot fully achieve maximum entropy. We propose Radial-VCReg, which augments VCReg with a radial Gaussianization loss that aligns feature norms with the Chi distribution-a defining property of high-dimensional Gaussians. We prove that Radial-VCReg transforms a broader class of distributions towards normality compared to VCReg and show on synthetic and real-world datasets that it consistently improves performance by reducing higher-order dependencies and promoting more diverse and informative representations.

Related papers

• Data-Adaptive Transformed Bilateral Tensor Low-Rank Representation for Clustering [17.14664384622625]
  We propose a novel bilateral low-rank representation model called TBTLRR.<n>It integrates a nuclear norm by arbitrary unitary unitary, allowing for more effective capture of global data.<n>It is able to exploit local correlations between image and latent correlations.
  arXiv  Detail & Related papers  (2025-10-22T23:25:44Z)
• Kernel VICReg for Self-Supervised Learning in Reproducing Kernel Hilbert Space [0.15293427903448018]
  Kernel VICReg is a self-supervised learning framework that lifts the VICReg objective into a Reproducing Kernel Hilbert Space.<n>We demonstrate that Kernel VICReg not only avoids representational collapse but also improves performance on tasks with complex or small-scale data.
  arXiv  Detail & Related papers  (2025-09-08T23:49:21Z)
• Fast State-Augmented Learning for Wireless Resource Allocation with Dual Variable Regression [83.27791109672927]
  We show how a state-augmented graph neural network (GNN) parametrization for the resource allocation policy circumvents the drawbacks of the ubiquitous dual subgradient methods.<n>Lagrangian maximizing state-augmented policies are learned during the offline training phase.<n>We prove a convergence result and an exponential probability bound on the excursions of the dual function (iterate) optimality gaps.
  arXiv  Detail & Related papers  (2025-06-23T15:20:58Z)
• Highly Adaptive Ridge [84.38107748875144]
  We propose a regression method that achieves a $n-2/3$ dimension-free L2 convergence rate in the class of right-continuous functions with square-integrable sectional derivatives. Har is exactly kernel ridge regression with a specific data-adaptive kernel based on a saturated zero-order tensor-product spline basis expansion. We demonstrate empirical performance better than state-of-the-art algorithms for small datasets in particular.
  arXiv  Detail & Related papers  (2024-10-03T17:06:06Z)
• Gradient-Based Spectral Embeddings of Random Dot Product Graphs [7.612218105739107]
  In this paper, we show how to better solve the embedding problem of the Random Dot Product Graph (RDPG) We develop a novel feasible optimization method in the resulting manifold. Our open-source algorithm implementations are scalable, and unlike the they are robust to missing edge data and can track slowly, latent positions from streaming graphs.
  arXiv  Detail & Related papers  (2023-07-25T21:09:55Z)
• Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression [53.15502562048627]
  Recent work has built the connection between self-supervised learning and the approximation of the top eigenspace of a graph Laplacian operator. This work delves into a statistical analysis of augmentation-based pretraining.
  arXiv  Detail & Related papers  (2023-06-01T15:18:55Z)
• VNE: An Effective Method for Improving Deep Representation by Manipulating Eigenvalue Distribution [2.5461557112299773]
  We propose to regularize von Neumann entropy(VNE) of representation. First, we demonstrate that the mathematical formulation of VNE is superior in effectively manipulating the eigenvalues of the representation autocorrelation matrix. In addition, we formally establish theoretical connections with rank, disentanglement, and isotropy of representation.
  arXiv  Detail & Related papers  (2023-04-04T01:03:32Z)
• Graph Polynomial Convolution Models for Node Classification of Non-Homophilous Graphs [52.52570805621925]
  We investigate efficient learning from higher-order graph convolution and learning directly from adjacency matrix for node classification. We show that the resulting model lead to new graphs and residual scaling parameter. We demonstrate that the proposed methods obtain improved accuracy for node-classification of non-homophilous parameters.
  arXiv  Detail & Related papers  (2022-09-12T04:46:55Z)
• Computer Vision Self-supervised Learning Methods on Time Series [1.30536490219656]
  Self-supervised learning (SSL) has had great success in both computer vision. Most of the current mainstream computer vision SSL frameworks are based on Siamese network architecture. We show that the computer vision SSL frameworks can be effective even for time series.
  arXiv  Detail & Related papers  (2021-09-02T08:45:53Z)
• SreaMRAK a Streaming Multi-Resolution Adaptive Kernel Algorithm [60.61943386819384]
  Existing implementations of KRR require that all the data is stored in the main memory. We propose StreaMRAK - a streaming version of KRR. We present a showcase study on two synthetic problems and the prediction of the trajectory of a double pendulum.
  arXiv  Detail & Related papers  (2021-08-23T21:03:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.