A Stable Neural Statistical Dependence Estimator for Autoencoder Feature Analysis

• URL: http://arxiv.org/abs/2603.11428v1
• Date: Thu, 12 Mar 2026 01:37:19 GMT
• Title: A Stable Neural Statistical Dependence Estimator for Autoencoder Feature Analysis
• Abstract summary: dependence measures like mutual information are ideal for analyzing autoencoders, but they can be ill-posed for deterministic, static, noise-free networks.<n>We adopt the variational (Gaussian) formulation that makes dependence among inputs, latents, and reconstructions measurable.<n>We propose a stable neural dependence estimator based on an orthonormal density-ratio decomposition.
• Score: 6.697324698653934
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Statistical dependence measures like mutual information is ideal for analyzing autoencoders, but it can be ill-posed for deterministic, static, noise-free networks. We adopt the variational (Gaussian) formulation that makes dependence among inputs, latents, and reconstructions measurable, and we propose a stable neural dependence estimator based on an orthonormal density-ratio decomposition. Unlike MINE, our method avoids input concatenation and product-of-marginals re-pairing, reducing computational cost and improving stability. We introduce an efficient NMF-like scalar objective and demonstrate empirically that assuming Gaussian noise to form an auxiliary variable enables meaningful dependence measurements and supports quantitative feature analysis, with a sequential convergence of singular values.

Related papers

• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• Semiparametrically Efficient Inference for Kernel Measures of Noise Heterogeneity [20.281748930003737]
  We develop semiparametrically efficient inference for kernel measures of noise heterogeneity in additive noise models.<n>We construct a novel one-step Hilbert-valued one-step estimator of the kernel cooperator between cocalibrates and residuals.<n>We show improved calibration and power relative to naive plug-in residual methods.
  arXiv  Detail & Related papers  (2026-05-26T18:01:09Z)
• SPPCSO: Adaptive Penalized Estimation Method for High-Dimensional Correlated Data [7.338284747998038]
  This work proposes the Single-Parametric Principal Component Selection Operator (SPPCSO), an innovative penalized estimation method.<n>It integrates single-parametric principal component regression and $L_1$ regularization to adaptively adjust the shrinkage factor by incorporating principal component information.<n>It achieves a balance between variable selection and coefficient estimation, ensuring model stability and robust estimation even in high-dimensional, high-noise environments.
  arXiv  Detail & Related papers  (2026-03-06T13:10:05Z)
• Profiling systematic uncertainties in Simulation-Based Inference with Factorizable Normalizing Flows [0.0]
  We propose a general framework for Simulation-Based Inference that efficiently profiles nuisance parameters.<n>We introduce Factorizable Normalizing Flows to model systematic variations as a parametrics of a nominal density.<n>We develop an amortized training strategy that learns the conditional dependence of the DoI on nuisance parameters in a single optimization process.<n>This allows for the simultaneous extraction of the underlying distribution and the robust profiling of nuisances.
  arXiv  Detail & Related papers  (2026-02-13T18:48:12Z)
• Equivariant Evidential Deep Learning for Interatomic Potentials [55.6997213490859]
  Uncertainty quantification is critical for assessing the reliability of machine learning interatomic potentials in molecular dynamics simulations.<n>Existing UQ approaches for MLIPs are often limited by high computational cost or suboptimal performance.<n>We propose textitEquivariant Evidential Deep Learning for Interatomic Potentials ($texte2$IP), a backbone-agnostic framework that models atomic forces and their uncertainty jointly.
  arXiv  Detail & Related papers  (2026-02-11T02:00:25Z)
• LVM-GP: Uncertainty-Aware PDE Solver via coupling latent variable model and Gaussian process [9.576396359649921]
  We propose a novel framework, termed LVM-GP, for uncertainty quantification in solving PDEs with noisy data.<n>The architecture consists of a confidence-aware encoder and a probabilistic decoder.
  arXiv  Detail & Related papers  (2025-07-30T09:00:39Z)
• Fragment size density estimator for shrinkage-induced fracture based on a physics-informed neural network [0.0]
  This paper presents a neural network (NN)-based solver for an integro-differential equation that models shrinkage-induced fragmentation.<n>The proposed method directly maps input parameters to the corresponding probability density function without numerically solving the governing equation.<n>It enables efficient evaluation of the density function in Monte Carlo simulations while maintaining accuracy comparable to or even exceeding that of conventional finite difference schemes.
  arXiv  Detail & Related papers  (2025-07-15T23:33:05Z)
• Regulating Model Reliance on Non-Robust Features by Smoothing Input Marginal Density [93.32594873253534]
  Trustworthy machine learning requires meticulous regulation of model reliance on non-robust features. We propose a framework to delineate and regulate such features by attributing model predictions to the input.
  arXiv  Detail & Related papers  (2024-07-05T09:16:56Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• Numerically Stable Sparse Gaussian Processes via Minimum Separation using Cover Trees [57.67528738886731]
  We study the numerical stability of scalable sparse approximations based on inducing points. For low-dimensional tasks such as geospatial modeling, we propose an automated method for computing inducing points satisfying these conditions.
  arXiv  Detail & Related papers  (2022-10-14T15:20:17Z)
• Instability, Computational Efficiency and Statistical Accuracy [101.32305022521024]
  We develop a framework that yields statistical accuracy based on interplay between the deterministic convergence rate of the algorithm at the population level, and its degree of (instability) when applied to an empirical object based on $n$ samples. We provide applications of our general results to several concrete classes of models, including Gaussian mixture estimation, non-linear regression models, and informative non-response models.
  arXiv  Detail & Related papers  (2020-05-22T22:30:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.