Exploration of Parameter Spaces Assisted by Machine Learning

• URL: http://arxiv.org/abs/2207.09959v1
• Date: Wed, 20 Jul 2022 15:09:16 GMT
• Title: Exploration of Parameter Spaces Assisted by Machine Learning
• Authors: A. Hammad, Myeonghun Park, Raymundo Ramos and Pankaj Saha
• Abstract summary: We show a variety of functions and classes that implement sampling procedures with improved exploration of the parameter space assisted by machine learning. In particular, we discuss two methods assisted by incorporating different machine learning models: regression and classification. The code used for this paper and instructions on how to use it are available on the web.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We showcase a variety of functions and classes that implement sampling procedures with improved exploration of the parameter space assisted by machine learning. Special attention is paid to setting sane defaults with the objective that adjustments required by different problems remain minimal. This collection of routines can be employed for different types of analysis, from finding bounds on the parameter space to accumulating samples in areas of interest. In particular, we discuss two methods assisted by incorporating different machine learning models: regression and classification. We show that a machine learning classifier can provide higher efficiency for exploring the parameter space. Also, we introduce a boosting technique to improve the slow convergence at the start of the process. The use of these routines is better explained with the help of a few examples that illustrate the type of results one can obtain. We also include examples of the code used to obtain the examples as well as descriptions of the adjustments that can be made to adapt the calculation to other problems. We finalize by showing the impact of these techniques when exploring the parameter space of the two Higgs doublet model that matches the measured Higgs Boson signal strength. The code used for this paper and instructions on how to use it are available on the web.

Related papers

• An Explainable Gaussian Process Auto-encoder for Tabular Data [6.360918504726019]
  We propose a novel method using a Gaussian process to construct the auto-encoder architecture for generating counterfactual samples.<n>The resulting model requires fewer learnable parameters and thus is less prone to overfitting.<n>We also introduce a novel density estimator that allows for searching for in-distribution samples.
  arXiv  Detail & Related papers  (2025-08-31T14:55:12Z)
• Localized Gaussians as Self-Attention Weights for Point Clouds Correspondence [92.07601770031236]
  We investigate semantically meaningful patterns in the attention heads of an encoder-only Transformer architecture. We find that fixing the attention weights not only accelerates the training process but also enhances the stability of the optimization.
  arXiv  Detail & Related papers  (2024-09-20T07:41:47Z)
• Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces [12.513822304719602]
  In settings with very few function evaluations, a successful application of BO may require transferring information from historical experiments. These related experiments may not have exactly the same tunable parameters (search spaces) We show that these two methods perform well on several benchmark problems.
  arXiv  Detail & Related papers  (2024-09-09T04:36:06Z)
• Parameter-Efficient and Memory-Efficient Tuning for Vision Transformer: A Disentangled Approach [87.8330887605381]
  We show how to adapt a pre-trained Vision Transformer to downstream recognition tasks with only a few learnable parameters. We synthesize a task-specific query with a learnable and lightweight module, which is independent of the pre-trained model. Our method achieves state-of-the-art performance under memory constraints, showcasing its applicability in real-world situations.
  arXiv  Detail & Related papers  (2024-07-09T15:45:04Z)
• Applications of model-aware reinforcement learning in Bayesian quantum metrology [0.6138671548064355]
  In Belliardo et al., arXiv:2312.16985 (2023), we introduced a procedure capable of optimizing a wide range of tasks in quantum metrology and estimation. We take a model-based approach to the optimisation where the physics describing the system is explicitly taken into account in the training. This exposition showcases the broad applicability of the method, which has been implemented in the qsensoropt library released on PyPI.
  arXiv  Detail & Related papers  (2024-03-08T22:48:07Z)
• An Efficient Algorithm for Clustered Multi-Task Compressive Sensing [60.70532293880842]
  Clustered multi-task compressive sensing is a hierarchical model that solves multiple compressive sensing tasks. The existing inference algorithm for this model is computationally expensive and does not scale well in high dimensions. We propose a new algorithm that substantially accelerates model inference by avoiding the need to explicitly compute these covariance matrices.
  arXiv  Detail & Related papers  (2023-09-30T15:57:14Z)
• Convolutional autoencoder-based multimodal one-class classification [80.52334952912808]
  One-class classification refers to approaches of learning using data from a single class only. We propose a deep learning one-class classification method suitable for multimodal data.
  arXiv  Detail & Related papers  (2023-09-25T12:31:18Z)
• Unbinned Profiled Unfolding [2.0813318162800707]
  Unfolding is an important procedure in particle physics experiments which corrects for detector effects. We propose a new machine learning-based unfolding method that results in an unbinned differential cross section.
  arXiv  Detail & Related papers  (2023-02-10T17:28:01Z)
• Active Learning with Expected Error Reduction [4.506537904404427]
  Expected Error Reduction (EER) has been shown to be an effective method for active learning. EER requires the model to be retrained for every candidate sample. In this paper we reformulate EER under the lens of Bayesian active learning.
  arXiv  Detail & Related papers  (2022-11-17T01:02:12Z)
• Low-variance estimation in the Plackett-Luce model via quasi-Monte Carlo sampling [58.14878401145309]
  We develop a novel approach to producing more sample-efficient estimators of expectations in the PL model. We illustrate our findings both theoretically and empirically using real-world recommendation data from Amazon Music and the Yahoo learning-to-rank challenge.
  arXiv  Detail & Related papers  (2022-05-12T11:15:47Z)
• A Framework and Benchmark for Deep Batch Active Learning for Regression [2.093287944284448]
  We study active learning methods that adaptively select batches of unlabeled data for labeling. We present a framework for constructing such methods out of (network-dependent) base kernels, kernel transformations, and selection methods. Our proposed method outperforms the state-of-the-art on our benchmark, scales to large data sets, and works out-of-the-box without adjusting the network architecture or training code.
  arXiv  Detail & Related papers  (2022-03-17T16:11:36Z)
• A Lagrangian Duality Approach to Active Learning [119.36233726867992]
  We consider the batch active learning problem, where only a subset of the training data is labeled. We formulate the learning problem using constrained optimization, where each constraint bounds the performance of the model on labeled samples. We show, via numerical experiments, that our proposed approach performs similarly to or better than state-of-the-art active learning methods.
  arXiv  Detail & Related papers  (2022-02-08T19:18:49Z)
• GENEOnet: A new machine learning paradigm based on Group Equivariant Non-Expansive Operators. An application to protein pocket detection [97.5153823429076]
  We introduce a new computational paradigm based on Group Equivariant Non-Expansive Operators. We test our method, called GENEOnet, on a key problem in drug design: detecting pockets on the surface of proteins that can host.
  arXiv  Detail & Related papers  (2022-01-31T11:14:51Z)
• Learning to Perform Downlink Channel Estimation in Massive MIMO Systems [72.76968022465469]
  We study downlink (DL) channel estimation in a Massive multiple-input multiple-output (MIMO) system. A common approach is to use the mean value as the estimate, motivated by channel hardening. We propose two novel estimation methods.
  arXiv  Detail & Related papers  (2021-09-06T13:42:32Z)
• Experimental Investigation and Evaluation of Model-based Hyperparameter Optimization [0.3058685580689604]
  This article presents an overview of theoretical and practical results for popular machine learning algorithms. The R package mlr is used as a uniform interface to the machine learning models.
  arXiv  Detail & Related papers  (2021-07-19T11:37:37Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• A concise method for feature selection via normalized frequencies [0.0]
  In this paper, a concise method is proposed for universal feature selection. The proposed method uses a fusion of the filter method and the wrapper method, rather than a combination of them. The evaluation results show that the proposed method outperformed several state-of-the-art related works in terms of accuracy, precision, recall, F-score and AUC.
  arXiv  Detail & Related papers  (2021-06-10T15:29:54Z)
• Improving Deep Learning Sound Events Classifiers using Gram Matrix Feature-wise Correlations [1.2891210250935146]
  In our method, we analyse all the activations of a generic CNN in order to produce feature representations using Gram Matrices. The proposed approach can be applied to any CNN and our experimental evaluation of four different architectures on two datasets demonstrated that our method consistently improves the baseline models.
  arXiv  Detail & Related papers  (2021-02-23T16:08:02Z)
• Multi-task Supervised Learning via Cross-learning [102.64082402388192]
  We consider a problem known as multi-task learning, consisting of fitting a set of regression functions intended for solving different tasks. In our novel formulation, we couple the parameters of these functions, so that they learn in their task specific domains while staying close to each other. This facilitates cross-fertilization in which data collected across different domains help improving the learning performance at each other task.
  arXiv  Detail & Related papers  (2020-10-24T21:35:57Z)
• Deducing neighborhoods of classes from a fitted model [68.8204255655161]
  In this article a new kind of interpretable machine learning method is presented. It can help to understand the partitioning of the feature space into predicted classes in a classification model using quantile shifts. Basically, real data points (or specific points of interest) are used and the changes of the prediction after slightly raising or decreasing specific features are observed.
  arXiv  Detail & Related papers  (2020-09-11T16:35:53Z)
• MetaDistiller: Network Self-Boosting via Meta-Learned Top-Down Distillation [153.56211546576978]
  In this work, we propose that better soft targets with higher compatibil-ity can be generated by using a label generator. We can employ the meta-learning technique to optimize this label generator. The experiments are conducted on two standard classificationbenchmarks, namely CIFAR-100 and ILSVRC2012.
  arXiv  Detail & Related papers  (2020-08-27T13:04:27Z)
• Variational Inference with Parameter Learning Applied to Vehicle Trajectory Estimation [20.41604350878599]
  We present parameter learning in a Gaussian variational inference setting using only noisy measurements. We demonstrate our technique using a 36km dataset consisting of a car using lidar to localize against a high-definition map.
  arXiv  Detail & Related papers  (2020-03-21T19:48:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.