On the Validation of Gibbs Algorithms: Training Datasets, Test Datasets and their Aggregation

• URL: http://arxiv.org/abs/2306.12380v1
• Date: Wed, 21 Jun 2023 16:51:50 GMT
• Title: On the Validation of Gibbs Algorithms: Training Datasets, Test Datasets and their Aggregation
• Authors: Samir M. Perlaza, I\~naki Esnaola, Gaetan Bisson, H. Vincent Poor
• Abstract summary: dependence on training data of the Gibbs algorithm (GA) is analytically characterized. This description enables the development of explicit expressions involving the training errors and test errors of GAs trained with different datasets.
• Score: 70.540936204654
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The dependence on training data of the Gibbs algorithm (GA) is analytically characterized. By adopting the expected empirical risk as the performance metric, the sensitivity of the GA is obtained in closed form. In this case, sensitivity is the performance difference with respect to an arbitrary alternative algorithm. This description enables the development of explicit expressions involving the training errors and test errors of GAs trained with different datasets. Using these tools, dataset aggregation is studied and different figures of merit to evaluate the generalization capabilities of GAs are introduced. For particular sizes of such datasets and parameters of the GAs, a connection between Jeffrey's divergence, training and test errors is established.

Related papers

• A novel approach to graph distinction through GENEOs and permutants [0.0]
  This paper explores the use of GENEOs for distinguishing $r$-regular graphs up to isomorphisms. Our experiments show that GENEOs offer a good compromise between efficiency and computational cost in comparing $r$-regular graphs. GENEOs could be a general-purpose approach to discriminative problems in Machine Learning.
  arXiv  Detail & Related papers  (2024-06-12T09:53:14Z)
• Grammatical Error Correction via Mixed-Grained Weighted Training [68.94921674855621]
  Grammatical Error Correction (GEC) aims to automatically correct grammatical errors in natural texts. MainGEC designs token-level and sentence-level training weights based on inherent discrepancies in accuracy and potential diversity of data annotation.
  arXiv  Detail & Related papers  (2023-11-23T08:34:37Z)
• Structure-Aware Code Vulnerability Analysis With Graph Neural Networks [0.0]
  This study explores the effectiveness of graph neural networks (GNNs) for vulnerability detection in software code. The primary focus is to evaluate the general applicability of GNNs in identifying vulnerable code segments and distinguishing these from their fixed versions. Experiments indicate that certain model configurations, such as the pruning of specific graph elements and the exclusion of certain types of code representation, significantly improve performance.
  arXiv  Detail & Related papers  (2023-07-21T09:35:29Z)
• Studying Generalization Through Data Averaging [0.0]
  We study train and test performance, as well as the generalization gap given by the mean of their difference over different data set samples. We predict some aspects about how the generalization gap and model train and test performance vary as a function of SGD noise.
  arXiv  Detail & Related papers  (2022-06-28T00:03:40Z)
• Modeling the Data-Generating Process is Necessary for Out-of-Distribution Generalization [23.302060306322506]
  Real-world data often has multiple distribution shifts over different attributes. No state-of-the-art DG algorithm performs consistently well on all shifts. We develop Causally Adaptive Constraint Minimization (CACM), an algorithm that uses knowledge about the data-generating process to adaptively identify and apply the correct independence constraints for regularization.
  arXiv  Detail & Related papers  (2022-06-15T22:35:06Z)
• Augmentation-Aware Self-Supervision for Data-Efficient GAN Training [68.81471633374393]
  Training generative adversarial networks (GANs) with limited data is challenging because the discriminator is prone to overfitting. We propose a novel augmentation-aware self-supervised discriminator that predicts the augmentation parameter of the augmented data. We compare our method with state-of-the-art (SOTA) methods using the class-conditional BigGAN and unconditional StyleGAN2 architectures.
  arXiv  Detail & Related papers  (2022-05-31T10:35:55Z)
• BCDAG: An R package for Bayesian structure and Causal learning of Gaussian DAGs [77.34726150561087]
  We introduce the R package for causal discovery and causal effect estimation from observational data. Our implementation scales efficiently with the number of observations and, whenever the DAGs are sufficiently sparse, the number of variables in the dataset. We then illustrate the main functions and algorithms on both real and simulated datasets.
  arXiv  Detail & Related papers  (2022-01-28T09:30:32Z)
• Learning Curves for SGD on Structured Features [23.40229188549055]
  We show that the geometry of the data in the induced feature space is crucial to accurately predict the test error throughout learning. We show that modeling the geometry of the data in the induced feature space is indeed crucial to accurately predict the test error throughout learning.
  arXiv  Detail & Related papers  (2021-06-04T20:48:20Z)
• On the Benefits of Invariance in Neural Networks [56.362579457990094]
  We show that training with data augmentation leads to better estimates of risk and thereof gradients, and we provide a PAC-Bayes generalization bound for models trained with data augmentation. We also show that compared to data augmentation, feature averaging reduces generalization error when used with convex losses, and tightens PAC-Bayes bounds.
  arXiv  Detail & Related papers  (2020-05-01T02:08:58Z)
• Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants [94.46276668068327]
  In [1], an ensemble of randomly projected linear discriminants is used to classify datasets. We develop a consistent estimator of the misclassification probability as an alternative to the computationally-costly cross-validation estimator. We also demonstrate the use of our estimator for tuning the projection dimension on both real and synthetic data.
  arXiv  Detail & Related papers  (2020-04-17T12:47:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.