Structured Basis Function Networks: Loss-Centric Multi-Hypothesis Ensembles with Controllable Diversity

• URL: http://arxiv.org/abs/2509.02792v1
• Date: Tue, 02 Sep 2025 19:53:43 GMT
• Title: Structured Basis Function Networks: Loss-Centric Multi-Hypothesis Ensembles with Controllable Diversity
• Authors: Alejandro Rodriguez Dominguez, Muhammad Shahzad, Xia Hong,
• Abstract summary: Existing approaches to predictive uncertainty rely on multi-hypothesis prediction, which promotes diversity but lacks principled aggregation.<n>The Structured Basis Function Network addresses this gap by linking multi-hypothesis prediction and ensembling through centroidal aggregation induced by Bregman divergences.<n>A tunable diversity mechanism provides parametric control of the bias-variance-diversity trade-off, connecting multi-hypothesis generalisation with loss-aware ensemble aggregation.
• Score: 46.60221265861393
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Existing approaches to predictive uncertainty rely either on multi-hypothesis prediction, which promotes diversity but lacks principled aggregation, or on ensemble learning, which improves accuracy but rarely captures the structured ambiguity. This implicitly means that a unified framework consistent with the loss geometry remains absent. The Structured Basis Function Network addresses this gap by linking multi-hypothesis prediction and ensembling through centroidal aggregation induced by Bregman divergences. The formulation applies across regression and classification by aligning predictions with the geometry of the loss, and supports both a closed-form least-squares estimator and a gradient-based procedure for general objectives. A tunable diversity mechanism provides parametric control of the bias-variance-diversity trade-off, connecting multi-hypothesis generalisation with loss-aware ensemble aggregation. Experiments validate this relation and use the mechanism to study the complexity-capacity-diversity trade-off across datasets of increasing difficulty with deep-learning predictors.

Related papers

• Multivariate Time Series Data Imputation via Distributionally Robust Regularization [2.3351357479046717]
  imputation compromised by mismatch between observed and true data distributions.<n>We propose the Distributionally Robust Regularized Imputer Objective (DRIO)<n>Experiments show DRIO consistently improves imputation under both missing-completely-at-random and missing-not-at-random settings.
  arXiv  Detail & Related papers  (2026-01-31T18:15:03Z)
• Random-Matrix-Induced Simplicity Bias in Over-parameterized Variational Quantum Circuits [72.0643009153473]
  We show that expressive variational ansatze enter a Haar-like universality class in which both observable expectation values and parameter gradients concentrate exponentially with system size.<n>As a consequence, the hypothesis class induced by such circuits collapses with high probability to a narrow family of near-constant functions.<n>We further show that this collapse is not unavoidable: tensor-structured VQCs, including tensor-network-based and tensor-hypernetwork parameterizations, lie outside the Haar-like universality class.
  arXiv  Detail & Related papers  (2026-01-05T08:04:33Z)
• Synergy Between Sufficient Changes and Sparse Mixing Procedure for Disentangled Representation Learning [32.482584125236016]
  Disentangled representation learning aims to uncover latent variables underlying the observed data.<n>Some approaches rely on sufficient changes on the distribution of latent variables indicated by auxiliary variables such as domain indices.<n>We propose an identifiability theory with less restrictive constraints regarding distribution changes and the sparse mixing procedure.
  arXiv  Detail & Related papers  (2025-03-01T22:21:37Z)
• EquiTabPFN: A Target-Permutation Equivariant Prior Fitted Networks [55.214444066134114]
  We design a fully target-equivariant architecture-ensuring permutation invariance via equivariant encoders, decoders, and a bi-attention mechanism.<n> Empirical evaluation on standard classification benchmarks shows that, on datasets with more classes than those seen during pre-training, our model matches or surpasses existing methods while incurring lower computational overhead.
  arXiv  Detail & Related papers  (2025-02-10T17:11:20Z)
• Bridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shift [44.708914058803224]
  We establish a new model-agnostic optimization framework for out-of-distribution generalization via multicalibration. We propose MC-Pseudolabel, a post-processing algorithm to achieve both extended multicalibration and out-of-distribution generalization.
  arXiv  Detail & Related papers  (2024-06-02T08:11:35Z)
• Out-of-distribution robustness for multivariate analysis via causal regularisation [4.487663958743944]
  We propose a regularisation strategy rooted in causality that ensures robustness against distribution shifts.<n>Building upon the anchor regression framework, we demonstrate how incorporating a straightforward regularisation term into the loss function of classical algorithms.<n>Our framework allows users to efficiently verify the compatibility of a loss function with the regularisation strategy.
  arXiv  Detail & Related papers  (2024-03-04T09:21:10Z)
• Structured Radial Basis Function Network: Modelling Diversity for Multiple Hypotheses Prediction [51.82628081279621]
  Multi-modal regression is important in forecasting nonstationary processes or with a complex mixture of distributions. A Structured Radial Basis Function Network is presented as an ensemble of multiple hypotheses predictors for regression problems. It is proved that this structured model can efficiently interpolate this tessellation and approximate the multiple hypotheses target distribution.
  arXiv  Detail & Related papers  (2023-09-02T01:27:53Z)
• Learning Linear Causal Representations from Interventions under General Nonlinear Mixing [52.66151568785088]
  We prove strong identifiability results given unknown single-node interventions without access to the intervention targets. This is the first instance of causal identifiability from non-paired interventions for deep neural network embeddings.
  arXiv  Detail & Related papers  (2023-06-04T02:32:12Z)
• Deep Anti-Regularized Ensembles provide reliable out-of-distribution uncertainty quantification [4.750521042508541]
  Deep ensemble often return overconfident estimates outside the training domain. We show that an ensemble of networks with large weights fitting the training data are likely to meet these two objectives. We derive a theoretical framework for this approach and show that the proposed optimization can be seen as a "water-filling" problem.
  arXiv  Detail & Related papers  (2023-04-08T15:25:12Z)
• Which Invariance Should We Transfer? A Causal Minimax Learning Approach [18.71316951734806]
  We present a comprehensive minimax analysis from a causal perspective. We propose an efficient algorithm to search for the subset with minimal worst-case risk. The effectiveness and efficiency of our methods are demonstrated on synthetic data and the diagnosis of Alzheimer's disease.
  arXiv  Detail & Related papers  (2021-07-05T09:07:29Z)
• General stochastic separation theorems with optimal bounds [68.8204255655161]
  Phenomenon of separability was revealed and used in machine learning to correct errors of Artificial Intelligence (AI) systems and analyze AI instabilities. Errors or clusters of errors can be separated from the rest of the data. The ability to correct an AI system also opens up the possibility of an attack on it, and the high dimensionality induces vulnerabilities caused by the same separability.
  arXiv  Detail & Related papers  (2020-10-11T13:12:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.