Best Subset Selection with Efficient Primal-Dual Algorithm

• URL: http://arxiv.org/abs/2207.02058v1
• Date: Tue, 5 Jul 2022 14:07:52 GMT
• Title: Best Subset Selection with Efficient Primal-Dual Algorithm
• Authors: Shaogang Ren, Guanhua Fang, Ping Li
• Abstract summary: Best subset selection is considered the gold standard' for many learning problems. In this paper, we investigate the dual forms of a family of $ell$-regularized problems. An efficient primal-dual method has been developed based on the primal and dual problem structures.
• Score: 24.568094642425837
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Best subset selection is considered the `gold standard' for many sparse learning problems. A variety of optimization techniques have been proposed to attack this non-convex and NP-hard problem. In this paper, we investigate the dual forms of a family of $\ell_0$-regularized problems. An efficient primal-dual method has been developed based on the primal and dual problem structures. By leveraging the dual range estimation along with the incremental strategy, our algorithm potentially reduces redundant computation and improves the solutions of best subset selection. Theoretical analysis and experiments on synthetic and real-world datasets validate the efficiency and statistical properties of the proposed solutions.

Related papers

• Dynamic Incremental Optimization for Best Subset Selection [15.8362578568708]
  Best subset selection is considered the gold standard for many learning problems. An efficient subset-dual algorithm is developed based on the primal and dual problem structures.
  arXiv  Detail & Related papers  (2024-02-04T02:26:40Z)
• Combining Kernelized Autoencoding and Centroid Prediction for Dynamic Multi-objective Optimization [3.431120541553662]
  This paper proposes a unified paradigm, which combines the kernelized autoncoding evolutionary search and the centriod-based prediction. The proposed method is compared with five state-of-the-art algorithms on a number of complex benchmark problems.
  arXiv  Detail & Related papers  (2023-12-02T00:24:22Z)
• Accelerating Cutting-Plane Algorithms via Reinforcement Learning Surrogates [49.84541884653309]
  A current standard approach to solving convex discrete optimization problems is the use of cutting-plane algorithms. Despite the existence of a number of general-purpose cut-generating algorithms, large-scale discrete optimization problems continue to suffer from intractability. We propose a method for accelerating cutting-plane algorithms via reinforcement learning.
  arXiv  Detail & Related papers  (2023-07-17T20:11:56Z)
• Linearization Algorithms for Fully Composite Optimization [61.20539085730636]
  This paper studies first-order algorithms for solving fully composite optimization problems convex compact sets. We leverage the structure of the objective by handling differentiable and non-differentiable separately, linearizing only the smooth parts.
  arXiv  Detail & Related papers  (2023-02-24T18:41:48Z)
• Solving integer multi-objective optimization problems using TOPSIS, Differential Evolution and Tabu Search [4.213427823201119]
  This paper presents a method to solve nonlinear integer multiobjective optimization problems. First, the problem is formulated using the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) Next, the Differential Evolution (DE) algorithm in its three versions (standard DE, DEGL, DEGL) are used as DE best and DEGL. Since the solutions found by the DE algorithms are continuous, the Tabu Search (TS) algorithm is employed to find integer solutions during the optimization process.
  arXiv  Detail & Related papers  (2022-04-05T23:59:33Z)
• Outlier-Robust Sparse Estimation via Non-Convex Optimization [73.18654719887205]
  We explore the connection between high-dimensional statistics and non-robust optimization in the presence of sparsity constraints. We develop novel and simple optimization formulations for these problems. As a corollary, we obtain that any first-order method that efficiently converges to station yields an efficient algorithm for these tasks.
  arXiv  Detail & Related papers  (2021-09-23T17:38:24Z)
• Bilevel Optimization: Convergence Analysis and Enhanced Design [63.64636047748605]
  Bilevel optimization is a tool for many machine learning problems. We propose a novel stoc-efficientgradient estimator named stoc-BiO.
  arXiv  Detail & Related papers  (2020-10-15T18:09:48Z)
• Meta-learning based Alternating Minimization Algorithm for Non-convex Optimization [9.774392581946108]
  We propose a novel solution for challenging non-problems of multiple variables. Our proposed approach is able to achieve effective iterations in cases while other methods would typically fail.
  arXiv  Detail & Related papers  (2020-09-09T10:45:00Z)
• A Riemannian Primal-dual Algorithm Based on Proximal Operator and its Application in Metric Learning [3.511851311025242]
  We propose a primal-dual algorithm to optimize the primal and dual variables iteratively. We prove convergence of the proposed algorithm and show its non-asymptotic convergence rate. Preliminary experimental results on an optimal fund selection problem in fund of funds management showed its efficacy.
  arXiv  Detail & Related papers  (2020-05-19T03:31:01Z)
• Optimal Randomized First-Order Methods for Least-Squares Problems [56.05635751529922]
  This class of algorithms encompasses several randomized methods among the fastest solvers for least-squares problems. We focus on two classical embeddings, namely, Gaussian projections and subsampled Hadamard transforms. Our resulting algorithm yields the best complexity known for solving least-squares problems with no condition number dependence.
  arXiv  Detail & Related papers  (2020-02-21T17:45:32Z)
• Extreme Algorithm Selection With Dyadic Feature Representation [78.13985819417974]
  We propose the setting of extreme algorithm selection (XAS) where we consider fixed sets of thousands of candidate algorithms. We assess the applicability of state-of-the-art AS techniques to the XAS setting and propose approaches leveraging a dyadic feature representation.
  arXiv  Detail & Related papers  (2020-01-29T09:40:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.