Diffusion-Guided Feature Selection via Nishimori Temperature: Noise-Based Spectral Embedding

• URL: http://arxiv.org/abs/2604.24692v1
• Date: Mon, 27 Apr 2026 16:53:40 GMT
• Title: Diffusion-Guided Feature Selection via Nishimori Temperature: Noise-Based Spectral Embedding
• Abstract summary: We propose Noise-Based Spectral Embedding (NBSE), a framework for selecting informative features from high-dimensional data without greedy search.<n>NBSE constructs a sparse similarity graph on the samples and identifies the Nishimori temperature $_N$ the critical inverse temperature at which the Bethe Hessian becomes singular.<n>By transposing the data matrix and applying NBSE in feature space, we obtain a one-dimensional spectral embedding that reveals groups of redundant or semantically related dimensions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We propose Noise-Based Spectral Embedding (NBSE), a physics-informed framework for selecting informative features from high-dimensional data without greedy search. NBSE constructs a sparse similarity graph on the samples and identifies the Nishimori temperature $β_N$ the critical inverse temperature at which the Bethe Hessian becomes singular. The corresponding smallest eigenvector captures the dominant mode of an intrinsically degree-corrected diffusion process, naturally reweighting nodes to prevent hub dominance. By transposing the data matrix and applying NBSE in feature space, we obtain a one-dimensional spectral embedding that reveals groups of redundant or semantically related dimensions; balanced binning then selects one representative per group. We prove that coloured Gaussian perturbations shift $β_N$ by at most $O(\barσ^2)$, guaranteeing robustness to measurement noise. Experiments on ImageNet embeddings from MobileNetV2 and EfficientNet-B4 show that NBSE preserves classification accuracy even under aggressive compression: on EfficientNet-B4 the accuracy drop is below $1\%$ when retaining only $30\%$ of features, outperforming ANOVA $F$-test and random selection by up to $6.8\%$.

Related papers

• Uncertainty-Aware Ensemble Deep Randomized Neural Networks for Classification [8.383154705659596]
  We propose intuitionistic fuzzy dRVFL (IF-dRVFL) and intuitionistic fuzzy edRVFL (IF-edRVFL) frameworks to enhance model robustness.<n>The proposed models unify intuitionistic fuzzy theory to exploit sample neighborhood information in the kernel space.<n>Experiments conducted on UCI and KEEL benchmark datasets demonstrate the superiority of the proposed IF-dRVFL and IF-edRVFL models over existing SOTA fuzzy and non-fuzzy approaches.
  arXiv  Detail & Related papers  (2026-08-07T21:37:56Z)
• Kohn-Sham Spectral Embedding on Sparse Graphs at the Nishimori Temperature for Image Classification [0.0]
  Kohn-Sham Spectral Embedding (KSSE)<n>An energy-based model replacing the top-layer of convolutional networks with a sparse-graph spectral embedding at the Nishimori temperature of an associated Random-Bond Ising Model.<n>On ImageNet-1000 with frozen EfficientNet-B4 features, KSSE achieves 88.93% Top-1 accuracy with 21.24M parameters-beating Swin-L (197M, 86.4-87.3%) and matching the lower end of ViT-H/14 (632M, 88.0-89.5%) with 10x and 30x fewer parameters.
  arXiv  Detail & Related papers  (2026-07-30T16:05:51Z)
• SEED: Targeted Data Selection by Weighted Independent Set [76.68391670109433]
  We develop a robust and scalable data selection pipeline dubbed SEED.<n>SEED consistently outperforms state-of-the-art methods on instruction tuning, visual instruction tuning, and semantic segmentation.
  arXiv  Detail & Related papers  (2026-05-15T07:26:54Z)
• Federated Language Models Under Bandwidth Budgets: Distillation Rates and Conformal Coverage [12.805268849262243]
  Training a language model on data scattered across bandwidth-limited nodes that cannot be centralized is a setting that arises in clinical networks, enterprise knowledge bases, and scientific consortia.<n>We study the regime in which data must remain distributed across nodes, and ask what statistical guarantees are in principle achievable under explicit bandwidth budgets.
  arXiv  Detail & Related papers  (2026-05-11T05:01:43Z)
• Spectral Tempering for Embedding Compression in Dense Passage Retrieval [17.660889990235656]
  We show that the optimal scaling strength $$ is not a global constant.<n>We propose Spectral Tempering, a learning-free method that derives an adaptive $(k)$ directly from the corpus eigenspectrum.
  arXiv  Detail & Related papers  (2026-03-19T10:01:32Z)
• Spectral Graph Clustering under Differential Privacy: Balancing Privacy, Accuracy, and Efficiency [53.98433419539793]
  We study the problem of spectral graph clustering under edge differential privacy (DP)<n>Specifically, we develop three mechanisms: (i) graph perturbation via randomized edge flipping combined with adjacency matrix shuffling, which enforces edge privacy; (ii) private graph projection with additive Gaussian noise in a lower-dimensional space to reduce dimensionality and computational complexity; and (iii) a noisy power iteration method that distributes Gaussian noise across iterations to ensure edge DP while maintaining convergence.
  arXiv  Detail & Related papers  (2025-10-08T15:30:27Z)
• Denoising Score Matching with Random Features: Insights on Diffusion Models from Precise Learning Curves [8.038035670274706]
  Empirical studies suggest that generalization and memorization are influenced by model complexity and the size of the training dataset.<n>We capture these behaviors by derivingally precise expressions for test and train errors of Denoising Score Matching (DSM) under a simple theoretical setting.<n>Our theoretical findings are consistent with the empirical observations.
  arXiv  Detail & Related papers  (2025-02-01T06:43:33Z)
• Robust Representation Consistency Model via Contrastive Denoising [83.47584074390842]
  randomized smoothing provides theoretical guarantees for certifying robustness against adversarial perturbations.<n> diffusion models have been successfully employed for randomized smoothing to purify noise-perturbed samples.<n>We reformulate the generative modeling task along the diffusion trajectories in pixel space as a discriminative task in the latent space.
  arXiv  Detail & Related papers  (2025-01-22T18:52:06Z)
• Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods [43.32546195968771]
  We study the data-dependent convergence and generalization behavior of gradient methods for neural networks with smooth activation.<n>Our results improve upon the shortcomings of the well-established Rademacher complexity-based bounds.<n>We show that a large step-size significantly improves upon the NTK regime's results in classifying the XOR distribution.
  arXiv  Detail & Related papers  (2024-10-13T21:49:29Z)
• SIRST-5K: Exploring Massive Negatives Synthesis with Self-supervised Learning for Robust Infrared Small Target Detection [53.19618419772467]
  Single-frame infrared small target (SIRST) detection aims to recognize small targets from clutter backgrounds. With the development of Transformer, the scale of SIRST models is constantly increasing. With a rich diversity of infrared small target data, our algorithm significantly improves the model performance and convergence speed.
  arXiv  Detail & Related papers  (2024-03-08T16:14:54Z)
• TD^2-Net: Toward Denoising and Debiasing for Dynamic Scene Graph Generation [76.24766055944554]
  We introduce a network named TD$2$-Net that aims at denoising and debiasing for dynamic SGG. TD$2$-Net outperforms the second-best competitors by 12.7 % on mean-Recall@10 for predicate classification.
  arXiv  Detail & Related papers  (2024-01-23T04:17:42Z)
• On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias [50.84569563188485]
  We show that gradient flow converges in direction when labels are determined by the sign of a target network with $r$ neurons. Our result may already hold for mild over- parameterization, where the width is $tildemathcalO(r)$ and independent of the sample size.
  arXiv  Detail & Related papers  (2022-05-18T16:57:10Z)
• Regularization by Misclassification in ReLU Neural Networks [3.288086999241324]
  We study the implicit bias of ReLU neural networks trained by a variant of SGD where at each step, the label is changed with probability $p$ to a random label. We show that label noise propels the network to a sparse solution in the following sense: for a typical input, a small fraction of neurons are active, and the firing pattern of the hidden layers is sparser.
  arXiv  Detail & Related papers  (2021-11-03T11:42:38Z)
• Deep learning for gravitational-wave data analysis: A resampling white-box approach [62.997667081978825]
  We apply Convolutional Neural Networks (CNNs) to detect gravitational wave (GW) signals of compact binary coalescences, using single-interferometer data from LIGO detectors. CNNs were quite precise to detect noise but not sensitive enough to recall GW signals, meaning that CNNs are better for noise reduction than generation of GW triggers.
  arXiv  Detail & Related papers  (2020-09-09T03:28:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.