Model identification and local linear convergence of coordinate descent

• URL: http://arxiv.org/abs/2010.11825v1
• Date: Thu, 22 Oct 2020 16:03:19 GMT
• Title: Model identification and local linear convergence of coordinate descent
• Authors: Quentin Klopfenstein and Quentin Bertrand and Alexandre Gramfort and Joseph Salmon and Samuel Vaiter
• Abstract summary: We show that cyclic coordinate descent achieves model identification in finite time for a wide class of functions. We also prove explicit local linear convergence rates for coordinate descent.
• Score: 74.87531444344381
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For composite nonsmooth optimization problems, Forward-Backward algorithm achieves model identification (e.g. support identification for the Lasso) after a finite number of iterations, provided the objective function is regular enough. Results concerning coordinate descent are scarcer and model identification has only been shown for specific estimators, the support-vector machine for instance. In this work, we show that cyclic coordinate descent achieves model identification in finite time for a wide class of functions. In addition, we prove explicit local linear convergence rates for coordinate descent. Extensive experiments on various estimators and on real datasets demonstrate that these rates match well empirical results.

Related papers

• Machine Learning for the identification of phase-transitions in interacting agent-based systems: a Desai-Zwanzig example [0.0]
  We propose a data-driven framework that pinpoints phase transitions for an agent-based model in its mean-field limit. To this end, we use the manifold learning algorithm Maps to identify a parsimonious set of data-driven latent variables. We then utilize a deep learning framework to obtain a conformal reparametrization of the data-driven coordinates.
  arXiv  Detail & Related papers  (2023-10-29T15:07:08Z)
• T-LoHo: A Bayesian Regularization Model for Structured Sparsity and Smoothness on Graphs [0.0]
  In graph-structured data, structured sparsity and smoothness tend to cluster together. We propose a new prior for high dimensional parameters with graphical relations. We use it to detect structured sparsity and smoothness simultaneously.
  arXiv  Detail & Related papers  (2021-07-06T10:10:03Z)
• SGD with Coordinate Sampling: Theory and Practice [0.0]
  MUSKETEER is an adaptive gradient descent algorithm for large scale problems. It samples the most coordinate coordinates on the noisy gradients and moves along the one direction yielding an important decrease of coordinates. Numerical experiments on both synthetic and real data examples confirm the effectiveness of MUSKETEER in large scale problems.
  arXiv  Detail & Related papers  (2021-05-25T10:37:50Z)
• Finding Geometric Models by Clustering in the Consensus Space [61.65661010039768]
  We propose a new algorithm for finding an unknown number of geometric models, e.g., homographies. We present a number of applications where the use of multiple geometric models improves accuracy. These include pose estimation from multiple generalized homographies; trajectory estimation of fast-moving objects.
  arXiv  Detail & Related papers  (2021-03-25T14:35:07Z)
• Articulated Shape Matching Using Laplacian Eigenfunctions and Unsupervised Point Registration [38.16866987817019]
  Spectral graph theory can be used to map these graphs onto lower dimensional spaces and match shapes by aligning their embeddings. We derive a new formulation that finds the best alignment between two congruent $K$-dimensional sets of points by selecting the best subset of eigenfunctions of the Laplacian matrix.
  arXiv  Detail & Related papers  (2020-12-14T08:49:25Z)
• Out-of-distribution Generalization via Partial Feature Decorrelation [72.96261704851683]
  We present a novel Partial Feature Decorrelation Learning (PFDL) algorithm, which jointly optimize a feature decomposition network and the target image classification model. The experiments on real-world datasets demonstrate that our method can improve the backbone model's accuracy on OOD image classification datasets.
  arXiv  Detail & Related papers  (2020-07-30T05:48:48Z)
• Making Affine Correspondences Work in Camera Geometry Computation [62.7633180470428]
  Local features provide region-to-region rather than point-to-point correspondences. We propose guidelines for effective use of region-to-region matches in the course of a full model estimation pipeline. Experiments show that affine solvers can achieve accuracy comparable to point-based solvers at faster run-times.
  arXiv  Detail & Related papers  (2020-07-20T12:07:48Z)
• Random extrapolation for primal-dual coordinate descent [61.55967255151027]
  We introduce a randomly extrapolated primal-dual coordinate descent method that adapts to sparsity of the data matrix and the favorable structures of the objective function. We show almost sure convergence of the sequence and optimal sublinear convergence rates for the primal-dual gap and objective values, in the general convex-concave case.
  arXiv  Detail & Related papers  (2020-07-13T17:39:35Z)
• Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants [94.46276668068327]
  In [1], an ensemble of randomly projected linear discriminants is used to classify datasets. We develop a consistent estimator of the misclassification probability as an alternative to the computationally-costly cross-validation estimator. We also demonstrate the use of our estimator for tuning the projection dimension on both real and synthetic data.
  arXiv  Detail & Related papers  (2020-04-17T12:47:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.