Using Distance Correlation for Efficient Bayesian Optimization

• URL: http://arxiv.org/abs/2102.08993v1
• Date: Wed, 17 Feb 2021 19:37:35 GMT
• Title: Using Distance Correlation for Efficient Bayesian Optimization
• Authors: Takuya Kanazawa
• Abstract summary: We propose a novel approach for Bayesian optimization, called $textsfGP-DC$. It balances exploration and exploitation automatically, and requires no manual parameter tuning. We evaluate $textsfGP-DC$ on a number of benchmark functions and observe that it outperforms state-of-the-art methods.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel approach for Bayesian optimization, called $\textsf{GP-DC}$, which combines Gaussian processes with distance correlation. It balances exploration and exploitation automatically, and requires no manual parameter tuning. We evaluate $\textsf{GP-DC}$ on a number of benchmark functions and observe that it outperforms state-of-the-art methods such as $\textsf{GP-UCB}$ and max-value entropy search, as well as the classical expected improvement heuristic. We also apply $\textsf{GP-DC}$ to optimize sequential integral observations with a variable integration range and verify its empirical efficiency on both synthetic and real-world datasets.

Related papers

• $f$-PO: Generalizing Preference Optimization with $f$-divergence Minimization [91.43730624072226]
  $f$-PO is a novel framework that generalizes and extends existing approaches. We conduct experiments on state-of-the-art language models using benchmark datasets.
  arXiv  Detail & Related papers  (2024-10-29T02:11:45Z)
• Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization [71.35604981129838]
  Traditional gradient-based bi-level optimization algorithms are ill-suited to meet the demands of large-scale applications. We introduce $(textFG)2textU$, which achieves an unbiased approximation of the meta gradient for bi-level optimization. $(textFG)2textU$ is inherently designed to support parallel computing, enabling it to effectively leverage large-scale distributed computing systems.
  arXiv  Detail & Related papers  (2024-06-20T08:21:52Z)
• A Specialized Semismooth Newton Method for Kernel-Based Optimal Transport [92.96250725599958]
  Kernel-based optimal transport (OT) estimators offer an alternative, functional estimation procedure to address OT problems from samples. We show that our SSN method achieves a global convergence rate of $O (1/sqrtk)$, and a local quadratic convergence rate under standard regularity conditions.
  arXiv  Detail & Related papers  (2023-10-21T18:48:45Z)
• Ensemble-based gradient inference for particle methods in optimization and sampling [2.9005223064604078]
  We propose an approach based on function evaluations and Bayesian inference to extract higher-order differential information. We suggest to use this information for the improvement of established ensemble-based numerical methods for optimization and sampling.
  arXiv  Detail & Related papers  (2022-09-23T09:21:35Z)
• Surrogate modeling for Bayesian optimization beyond a single Gaussian process [62.294228304646516]
  We propose a novel Bayesian surrogate model to balance exploration with exploitation of the search space. To endow function sampling with scalability, random feature-based kernel approximation is leveraged per GP model. To further establish convergence of the proposed EGP-TS to the global optimum, analysis is conducted based on the notion of Bayesian regret.
  arXiv  Detail & Related papers  (2022-05-27T16:43:10Z)
• DoCoM: Compressed Decentralized Optimization with Near-Optimal Sample Complexity [25.775517797956237]
  This paper proposes the Doubly Compressed Momentum-assisted tracking algorithm $ttDoCoM$ for communication. We show that our algorithm outperforms several state-of-the-art algorithms in practice.
  arXiv  Detail & Related papers  (2022-02-01T07:27:34Z)
• Bayesian Optimistic Optimisation with Exponentially Decaying Regret [58.02542541410322]
  The current practical BO algorithms have regret bounds ranging from $mathcalO(fraclogNsqrtN)$ to $mathcal O(e-sqrtN)$, where $N$ is the number of evaluations. This paper explores the possibility of improving the regret bound in the noiseless setting by intertwining concepts from BO and tree-based optimistic optimisation. We propose the BOO algorithm, a first practical approach which can achieve an exponential regret bound with order $mathcal O(N-sqrt
  arXiv  Detail & Related papers  (2021-05-10T13:07:44Z)
• Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence [66.83161885378192]
  Area under ROC (AUROC) and precision-recall curves (AUPRC) are common metrics for evaluating classification performance for imbalanced problems. We propose a technical method to optimize AUPRC for deep learning.
  arXiv  Detail & Related papers  (2021-04-18T06:22:21Z)
• Efficient Bayesian Optimization using Multiscale Graph Correlation [0.0]
  We propose a new approach to Bayesian optimization called GP-MGC. We present our evaluation of GP-MGC in applications involving both synthetic benchmark functions and real-world datasets.
  arXiv  Detail & Related papers  (2021-03-17T04:35:09Z)
• Optimistic search strategy: Change point detection for large-scale data via adaptive logarithmic queries [1.3212010735248336]
  Change point detection is often formulated as a search for the maximum of a gain function describing improved fits when segmenting the data. We propose optimistic search strategies with $O(log T)$ exploiting specific structure of the gain function.
  arXiv  Detail & Related papers  (2020-10-20T11:09:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.