On the Effects of Heterogeneous Errors on Multi-fidelity Bayesian Optimization

• URL: http://arxiv.org/abs/2309.02771v1
• Date: Wed, 6 Sep 2023 06:26:21 GMT
• Title: On the Effects of Heterogeneous Errors on Multi-fidelity Bayesian Optimization
• Authors: Zahra Zanjani Foumani, Amin Yousefpour, Mehdi Shishehbor, Ramin Bostanabad
• Abstract summary: We propose an MF emulation method that learns a noise model for each data source. We illustrate the performance of our method through analytical examples and engineering problems on materials design.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bayesian optimization (BO) is a sequential optimization strategy that is increasingly employed in a wide range of areas including materials design. In real world applications, acquiring high-fidelity (HF) data through physical experiments or HF simulations is the major cost component of BO. To alleviate this bottleneck, multi-fidelity (MF) methods are used to forgo the sole reliance on the expensive HF data and reduce the sampling costs by querying inexpensive low-fidelity (LF) sources whose data are correlated with HF samples. However, existing multi-fidelity BO (MFBO) methods operate under the following two assumptions that rarely hold in practical applications: (1) LF sources provide data that are well correlated with the HF data on a global scale, and (2) a single random process can model the noise in the fused data. These assumptions dramatically reduce the performance of MFBO when LF sources are only locally correlated with the HF source or when the noise variance varies across the data sources. In this paper, we dispense with these incorrect assumptions by proposing an MF emulation method that (1) learns a noise model for each data source, and (2) enables MFBO to leverage highly biased LF sources which are only locally correlated with the HF source. We illustrate the performance of our method through analytical examples and engineering problems on materials design.

Related papers

• Collaboratively Learning Federated Models from Noisy Decentralized Data [21.3209961590772]
  Federated learning (FL) has emerged as a prominent method for collaboratively training machine learning models using local data from edge devices. We focus on addressing the problem of noisy data in the input space, an under-explored area compared to the label noise. We propose a noise-aware FL aggregation method, namely Federated Noise-Sifting (FedNS), which can be used as a plug-in approach in conjunction with widely used FL strategies.
  arXiv  Detail & Related papers  (2024-09-03T18:00:51Z)
• Semi-Federated Learning: Convergence Analysis and Optimization of A Hybrid Learning Framework [70.83511997272457]
  We propose a semi-federated learning (SemiFL) paradigm to leverage both the base station (BS) and devices for a hybrid implementation of centralized learning (CL) and FL. We propose a two-stage algorithm to solve this intractable problem, in which we provide the closed-form solutions to the beamformers.
  arXiv  Detail & Related papers  (2023-10-04T03:32:39Z)
• Mitigating the Alignment Tax of RLHF [76.4300447532456]
  aligning LLMs under Reinforcement Learning with Human Feedback can lead to forgetting pretrained abilities, also known as the alignment tax. We propose model averaging to maximize alignment performance while incurring minimal alignment tax. We validate HMA's performance across a range of RLHF algorithms over OpenLLaMA-3B and further extend our findings to Mistral-7B.
  arXiv  Detail & Related papers  (2023-09-12T14:16:54Z)
• Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity [72.85248553787538]
  This paper focuses on performance analysis and optimization for wireless FL, considering data heterogeneity, combined with wireless resource allocation. We formulate the loss function minimization problem, under constraints on long-term energy consumption and latency, and jointly optimize client scheduling, resource allocation, and the number of local training epochs (CRE) Experiments on real-world datasets demonstrate that the proposed algorithm outperforms other benchmarks in terms of the learning accuracy and energy consumption.
  arXiv  Detail & Related papers  (2023-08-04T04:18:01Z)
• Multi-Source to Multi-Target Decentralized Federated Domain Adaptation [15.681197161658835]
  In this paper, we focus on varying quantities/distributions of labeled and unlabeled data across devices. We develop a decentralized federated domain adaptation methodology which considers the transfer of ML models from devices with high quality labeled data to devices with low quality or unlabeled data. Our methodology, Source-Target Determination and Link Formation (ST-LF), optimize both (i) classification of devices into sources and targets and (ii) source-target link formation.
  arXiv  Detail & Related papers  (2023-04-24T19:57:13Z)
• Probabilistic Neural Data Fusion for Learning from an Arbitrary Number of Multi-fidelity Data Sets [0.0]
  In this paper, we employ neural networks (NNs) for data fusion in scenarios where data is very scarce. We introduce a unique NN architecture that converts MF modeling into a nonlinear manifold learning problem. Our approach provides a high predictive power while quantifying various sources uncertainties.
  arXiv  Detail & Related papers  (2023-01-30T20:27:55Z)
• Multi-Fidelity Cost-Aware Bayesian Optimization [0.0]
  An increasingly popular strategy in Bayesian optimization (BO) is to forgo the sole reliance on high-fidelity data and instead use an ensemble of information sources which provide inexpensive low-fidelity data. Here, we propose a multi-fidelity cost-aware BO framework that dramatically outperforms the state-of-the-art technologies in terms of efficiency, consistency, and robustness.
  arXiv  Detail & Related papers  (2022-11-04T20:07:24Z)
• DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup [58.894901088797376]
  federated learning has emerged as a promising approach for training a global model using data from multiple organizations without leaking their raw data. We propose a general framework to solve the above two challenges simultaneously. We provide comprehensive theoretical analysis including robustness analysis, convergence analysis, and generalization ability.
  arXiv  Detail & Related papers  (2022-04-16T08:08:29Z)
• GAN-MDF: A Method for Multi-fidelity Data Fusion in Digital Twins [82.71367011801242]
  Internet of Things (IoT) collects real-time data of physical systems, such as smart factory, intelligent robot and healtcare system. High-fidelity (HF) responses describe the system of interest accurately but are computed costly. Low-fidelity (LF) responses have a low computational cost but could not meet the required accuracy. We propose a novel generative adversarial network for MDF in digital twins (GAN-MDF)
  arXiv  Detail & Related papers  (2021-06-24T06:40:35Z)
• Adaptive Multi-View ICA: Estimation of noise levels for optimal inference [65.94843987207445]
  Adaptive multiView ICA (AVICA) is a noisy ICA model where each view is a linear mixture of shared independent sources with additive noise on the sources. On synthetic data, AVICA yields better sources estimates than other group ICA methods thanks to its explicit MMSE estimator. On real magnetoencephalograpy (MEG) data, we provide evidence that the decomposition is less sensitive to sampling noise and that the noise variance estimates are biologically plausible.
  arXiv  Detail & Related papers  (2021-02-22T13:10:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.