Avoiding Traps in Nonconvex Problems

• URL: http://arxiv.org/abs/2106.05206v1
• Date: Wed, 9 Jun 2021 16:47:53 GMT
• Title: Avoiding Traps in Nonconvex Problems
• Abstract summary: Iterative projection methods may become trapped at non-solutions when non- constraint sets are non-constrained. Two kinds of parameters are available to help avoid this feature.
• Score: 7.056222499095849
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Iterative projection methods may become trapped at non-solutions when the constraint sets are nonconvex. Two kinds of parameters are available to help avoid this behavior and this study gives examples of both. The first kind of parameter, called a hyperparameter, includes any kind of parameter that appears in the definition of the iteration rule itself. The second kind comprises metric parameters in the definition of the constraint sets, a feature that arises when the problem to be solved has two or more kinds of variables. Through examples we show the importance of properly tuning both kinds of parameters and offer heuristic interpretations of the observed behavior.

Related papers

• Two-parameter classes of exactly solvable quantum systems [0.0]
  We introduce exactly-solvable novel systems whose Hamiltonian operators could be represented by tridiagonal symmetric matrices in some orthonormal basis set.<n>We give several illustrative examples of these systems with continuous and/or discrete energy spectra.<n>A curious phenomenon is observed where bound states and/or resonances are induced in a system with pure continuous spectrum if the two parameters in the initial values exceed certain critical limits.
  arXiv  Detail & Related papers  (2026-05-10T23:29:07Z)
• Towards Fully Parameter-Free Stochastic Optimization: Grid Search with Self-Bounding Analysis [44.258546874111566]
  Search-free optimization aims to design algorithms that are to the underlying problem parameters.<n>We propose a powerful and general grid search framework, named scGrasp, with a novel self-bounding analysis technique.
  arXiv  Detail & Related papers  (2026-04-18T07:32:57Z)
• Continual Adaptation: Environment-Conditional Parameter Generation for Object Detection in Dynamic Scenarios [54.58186816693791]
  environments constantly change over time and space, posing significant challenges for object detectors trained based on a closed-set assumption.<n>We propose a new mechanism, converting the fine-tuning process to a specific- parameter generation.<n>In particular, we first design a dual-path LoRA-based domain-aware adapter that disentangles features into domain-invariant and domain-specific components.
  arXiv  Detail & Related papers  (2025-06-30T17:14:12Z)
• Quantum Parameter Estimation Uncertainty Relation [0.6445605125467574]
  We derive an estimation uncertainty relation that quantifies how measurement incompatibility and parameter correlation limit estimation precision.<n>Our results reveal that a geometric perspective of the parameter space offers a powerful approach for addressing multi parameter estimation challenges.
  arXiv  Detail & Related papers  (2025-06-18T11:14:09Z)
• Achieving the Multi-parameter Quantum Cramér-Rao Bound with Antiunitary Symmetry [18.64293022108985]
  We propose a novel and comprehensive approach to optimize the parameters encoding strategies with the aid of antiunitary symmetry. The results showcase the simultaneous achievement of ultimate precision for multiple parameters without any trade-off.
  arXiv  Detail & Related papers  (2024-11-22T13:37:43Z)
• Scaling Exponents Across Parameterizations and Optimizers [94.54718325264218]
  We propose a new perspective on parameterization by investigating a key assumption in prior work. Our empirical investigation includes tens of thousands of models trained with all combinations of threes. We find that the best learning rate scaling prescription would often have been excluded by the assumptions in prior work.
  arXiv  Detail & Related papers  (2024-07-08T12:32:51Z)
• Overparameterized Multiple Linear Regression as Hyper-Curve Fitting [0.0]
  It is proven that a linear model will produce exact predictions even in the presence of nonlinear dependencies that violate the model assumptions. The hyper-curve approach is especially suited for the regularization of problems with noise in predictor variables and can be used to remove noisy and "improper" predictors from the model.
  arXiv  Detail & Related papers  (2024-04-11T15:43:11Z)
• How Free is Parameter-Free Stochastic Optimization? [29.174036532175855]
  We study the problem of parameter-free optimization, inquiring whether, and under what conditions, do fully parameter-free methods exist. Existing methods can only be considered partially'' parameter-free, as they require some non-trivial knowledge of the true problem parameters. We demonstrate that a simple hyper search technique results in a fully parameter-free method that outperforms more sophisticated state-the-art algorithms.
  arXiv  Detail & Related papers  (2024-02-05T15:51:49Z)
• Online Constraint Tightening in Stochastic Model Predictive Control: A Regression Approach [49.056933332667114]
  No analytical solutions exist for chance-constrained optimal control problems. We propose a data-driven approach for learning the constraint-tightening parameters online during control. Our approach yields constraint-tightening parameters that tightly satisfy the chance constraints.
  arXiv  Detail & Related papers  (2023-10-04T16:22:02Z)
• A Variational Inference Approach to Inverse Problems with Gamma Hyperpriors [60.489902135153415]
  This paper introduces a variational iterative alternating scheme for hierarchical inverse problems with gamma hyperpriors. The proposed variational inference approach yields accurate reconstruction, provides meaningful uncertainty quantification, and is easy to implement.
  arXiv  Detail & Related papers  (2021-11-26T06:33:29Z)
• On the Parameter Combinations That Matter and on Those That do Not [0.0]
  We present a data-driven approach to characterizing nonidentifiability of a model's parameters. By employing Diffusion Maps and their extensions, we discover the minimal combinations of parameters required to characterize the dynamic output behavior.
  arXiv  Detail & Related papers  (2021-10-13T13:46:23Z)
• Online Hyperparameter Meta-Learning with Hypergradient Distillation [59.973770725729636]
  gradient-based meta-learning methods assume a set of parameters that do not participate in inner-optimization. We propose a novel HO method that can overcome these limitations, by approximating the second-order term with knowledge distillation.
  arXiv  Detail & Related papers  (2021-10-06T05:14:53Z)
• Gaussian Process Uniform Error Bounds with Unknown Hyperparameters for Safety-Critical Applications [71.23286211775084]
  We introduce robust Gaussian process uniform error bounds in settings with unknown hyper parameters. Our approach computes a confidence region in the space of hyper parameters, which enables us to obtain a probabilistic upper bound for the model error. Experiments show that the bound performs significantly better than vanilla and fully Bayesian processes.
  arXiv  Detail & Related papers  (2021-09-06T17:10:01Z)
• Implicit differentiation of Lasso-type models for hyperparameter optimization [82.73138686390514]
  We introduce an efficient implicit differentiation algorithm, without matrix inversion, tailored for Lasso-type problems. Our approach scales to high-dimensional data by leveraging the sparsity of the solutions.
  arXiv  Detail & Related papers  (2020-02-20T18:43:42Z)
• One from many: Estimating a function of many parameters [0.0]
  Many parameters of a process are unknown; estimate a specific linear combination of these parameters without having the ability to control any of the parameters. Geometric reasoning demonstrates the conditions, necessary and sufficient, for saturating the fundamental and attainable quantum-process bound.
  arXiv  Detail & Related papers  (2020-02-07T16:55:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.