Bandwidth Selectors on Semiparametric Bayesian Networks

• URL: http://arxiv.org/abs/2506.16844v1
• Date: Fri, 20 Jun 2025 08:48:05 GMT
• Title: Bandwidth Selectors on Semiparametric Bayesian Networks
• Authors: Victor Alejandre, Concha Bielza, Pedro Larrañaga,
• Abstract summary: Semiparametric Bayesian networks (SPBNs) integrate parametric and non-parametric probabilistic models.<n>In particular, kernel density estimators (KDEs) are employed for the non-parametric component.<n>This paper first establishes the theoretical framework for the application of state-of-the-art bandwidth selectors.
• Score: 3.6998629873543125
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Semiparametric Bayesian networks (SPBNs) integrate parametric and non-parametric probabilistic models, offering flexibility in learning complex data distributions from samples. In particular, kernel density estimators (KDEs) are employed for the non-parametric component. Under the assumption of data normality, the normal rule is used to learn the bandwidth matrix for the KDEs in SPBNs. This matrix is the key hyperparameter that controls the trade-off between bias and variance. However, real-world data often deviates from normality, potentially leading to suboptimal density estimation and reduced predictive performance. This paper first establishes the theoretical framework for the application of state-of-the-art bandwidth selectors and subsequently evaluates their impact on SPBN performance. We explore the approaches of cross-validation and plug-in selectors, assessing their effectiveness in enhancing the learning capability and applicability of SPBNs. To support this investigation, we have extended the open-source package PyBNesian for SPBNs with the additional bandwidth selection techniques and conducted extensive experimental analyses. Our results demonstrate that the proposed bandwidth selectors leverage increasing information more effectively than the normal rule, which, despite its robustness, stagnates with more data. In particular, unbiased cross-validation generally outperforms the normal rule, highlighting its advantage in high sample size scenarios.

Related papers

• Hybrid Batch Normalisation: Resolving the Dilemma of Batch Normalisation in Federated Learning [34.07540860626276]
  Batch Normalisation (BN) is widely used in deep neural network training to harmonise the input-output distributions for each batch of data.<n>In this work, we develop a customised normalisation approach, Hybrid Batch Normalisation (HBN)<n>HBN separates the update of statistical parameters from that of learnable parameters, obtaining unbiased estimates of global statistical parameters in distributed scenarios.
  arXiv  Detail & Related papers  (2025-05-28T01:46:34Z)
• Rethinking Clustered Federated Learning in NOMA Enhanced Wireless Networks [60.09912912343705]
  This study explores the benefits of integrating the novel clustered federated learning (CFL) approach with non-independent and identically distributed (non-IID) datasets. A detailed theoretical analysis of the generalization gap that measures the degree of non-IID in the data distribution is presented. Solutions to address the challenges posed by non-IID conditions are proposed with the analysis of the properties.
  arXiv  Detail & Related papers  (2024-03-05T17:49:09Z)
• Revisiting the Dataset Bias Problem from a Statistical Perspective [72.94990819287551]
  We study the "dataset bias" problem from a statistical standpoint. We identify the main cause of the problem as the strong correlation between a class attribute u and a non-class attribute b. We propose to mitigate dataset bias via either weighting the objective of each sample n by frac1p(u_n|b_n) or sampling that sample with a weight proportional to frac1p(u_n|b_n).
  arXiv  Detail & Related papers  (2024-02-05T22:58:06Z)
• Marginal Debiased Network for Fair Visual Recognition [59.05212866862219]
  We propose a novel marginal debiased network (MDN) to learn debiased representations. Our MDN can achieve a remarkable performance on under-represented samples.
  arXiv  Detail & Related papers  (2024-01-04T08:57:09Z)
• Skew-Probabilistic Neural Networks for Learning from Imbalanced Data [3.233103072575564]
  This paper introduces an imbalanced data-oriented classifier using probabilistic neural networks (PNN) with a skew-normal kernel function to address this major challenge.<n>By leveraging the skew-normal distribution, which offers increased flexibility, our proposed Skew-Probabilistic Neural Networks (SkewPNN) can better represent underlying class densities.<n>Real-data analysis on several datasets shows that SkewPNN and BA-SkewPNN substantially outperform most state-of-the-art machine-learning methods for both balanced and imbalanced datasets.
  arXiv  Detail & Related papers  (2023-12-10T13:12:55Z)
• Optimal Regularization for a Data Source [8.38093977965175]
  It is common to augment criteria that enforce data fidelity with a regularizer that promotes quantity in the solution. In this paper we seek a systematic understanding of the power and the limitations of convex regularization.
  arXiv  Detail & Related papers  (2022-12-27T20:11:59Z)
• FaDIn: Fast Discretized Inference for Hawkes Processes with General Parametric Kernels [82.53569355337586]
  This work offers an efficient solution to temporal point processes inference using general parametric kernels with finite support. The method's effectiveness is evaluated by modeling the occurrence of stimuli-induced patterns from brain signals recorded with magnetoencephalography (MEG) Results show that the proposed approach leads to an improved estimation of pattern latency than the state-of-the-art.
  arXiv  Detail & Related papers  (2022-10-10T12:35:02Z)
• Theroretical Insight into Batch Normalization: Data Dependant Auto-Tuning of Regularization Rate [1.6447597767676658]
  Batch normalization is widely used in deep learning to normalize intermediate activations. This paper brings out the data-dependent auto-tuning of the regularization parameter by textbfBN with analytical proofs.
  arXiv  Detail & Related papers  (2022-09-15T19:51:02Z)
• Pseudo-Spherical Contrastive Divergence [119.28384561517292]
  We propose pseudo-spherical contrastive divergence (PS-CD) to generalize maximum learning likelihood of energy-based models. PS-CD avoids the intractable partition function and provides a generalized family of learning objectives.
  arXiv  Detail & Related papers  (2021-11-01T09:17:15Z)
• Semiparametric Bayesian Networks [5.205440005969871]
  We introduce semiparametric Bayesian networks that combine parametric and nonparametric conditional probability distributions. Their aim is to incorporate the bounded complexity of parametric models and the flexibility of nonparametric ones.
  arXiv  Detail & Related papers  (2021-09-07T11:47:32Z)
• Statistical Mechanical Analysis of Neural Network Pruning [6.029526715675584]
  We show that DPP based node pruning method is notably superior to competing approaches when tested on real datasets. We use our theoretical setup to prove this finding and show that even the baseline random edge pruning method performs better than the DPP node pruning method.
  arXiv  Detail & Related papers  (2020-06-30T09:15:25Z)
• Dependency Aware Filter Pruning [74.69495455411987]
  Pruning a proportion of unimportant filters is an efficient way to mitigate the inference cost. Previous work prunes filters according to their weight norms or the corresponding batch-norm scaling factors. We propose a novel mechanism to dynamically control the sparsity-inducing regularization so as to achieve the desired sparsity.
  arXiv  Detail & Related papers  (2020-05-06T07:41:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.