A generalized linear joint trained framework for semi-supervised learning of sparse features

• URL: http://arxiv.org/abs/2006.01671v2
• Date: Fri, 2 Oct 2020 12:24:09 GMT
• Title: A generalized linear joint trained framework for semi-supervised learning of sparse features
• Authors: Juan C. Laria and Line H. Clemmensen and Bjarne K. Ersb{\o}ll
• Abstract summary: The elastic-net is among the most widely used types of regularization algorithms. This paper introduces a novel solution for semi-supervised learning of sparse features in the context of generalized linear model estimation.
• Score: 4.511923587827301
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The elastic-net is among the most widely used types of regularization algorithms, commonly associated with the problem of supervised generalized linear model estimation via penalized maximum likelihood. Its nice properties originate from a combination of $\ell_1$ and $\ell_2$ norms, which endow this method with the ability to select variables taking into account the correlations between them. In the last few years, semi-supervised approaches, that use both labeled and unlabeled data, have become an important component in the statistical research. Despite this interest, however, few researches have investigated semi-supervised elastic-net extensions. This paper introduces a novel solution for semi-supervised learning of sparse features in the context of generalized linear model estimation: the generalized semi-supervised elastic-net (s2net), which extends the supervised elastic-net method, with a general mathematical formulation that covers, but is not limited to, both regression and classification problems. We develop a flexible and fast implementation for s2net in R, and its advantages are illustrated using both real and synthetic data sets.

Related papers

• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  This paper presents a succinct derivation of the training and generalization performance of a variety of high-dimensional ridge regression models. We provide an introduction and review of recent results on these topics, aimed at readers with backgrounds in physics and deep learning.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• Provably tuning the ElasticNet across instances [53.0518090093538]
  We consider the problem of tuning the regularization parameters of Ridge regression, LASSO, and the ElasticNet across multiple problem instances. Our results are the first general learning-theoretic guarantees for this important class of problems.
  arXiv  Detail & Related papers  (2022-07-20T21:22:40Z)
• Intrinsic dimensionality and generalization properties of the $\mathcal{R}$-norm inductive bias [4.37441734515066]
  The $mathcalR$-norm is the basis of an inductive bias for two-layer neural networks. We find that these interpolants are intrinsically multivariate functions, even when there are ridge functions that fit the data.
  arXiv  Detail & Related papers  (2022-06-10T18:33:15Z)
• Harmless interpolation in regression and classification with structured features [21.064512161584872]
  Overparametrized neural networks tend to perfectly fit noisy training data yet generalize well on test data. We present a general and flexible framework for upper bounding regression and classification risk in a reproducing kernel Hilbert space.
  arXiv  Detail & Related papers  (2021-11-09T15:12:26Z)
• The Interplay Between Implicit Bias and Benign Overfitting in Two-Layer Linear Networks [51.1848572349154]
  neural network models that perfectly fit noisy data can generalize well to unseen test data. We consider interpolating two-layer linear neural networks trained with gradient flow on the squared loss and derive bounds on the excess risk.
  arXiv  Detail & Related papers  (2021-08-25T22:01:01Z)
• Dimension Free Generalization Bounds for Non Linear Metric Learning [61.193693608166114]
  We provide uniform generalization bounds for two regimes -- the sparse regime, and a non-sparse regime. We show that by relying on a different, new property of the solutions, it is still possible to provide dimension free generalization guarantees.
  arXiv  Detail & Related papers  (2021-02-07T14:47:00Z)
• Structure Learning in Inverse Ising Problems Using $\ell_2$-Regularized Linear Estimator [8.89493507314525]
  We show that despite the model mismatch, one can perfectly identify the network structure using naive linear regression without regularization. We propose a two-stage estimator: In the first stage, the ridge regression is used and the estimates are pruned by a relatively small threshold. This estimator with the appropriate regularization coefficient and thresholds is shown to achieve the perfect identification of the network structure even in $0M/N1$.
  arXiv  Detail & Related papers  (2020-08-19T09:11:33Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)
• Slice Sampling for General Completely Random Measures [74.24975039689893]
  We present a novel Markov chain Monte Carlo algorithm for posterior inference that adaptively sets the truncation level using auxiliary slice variables. The efficacy of the proposed algorithm is evaluated on several popular nonparametric models.
  arXiv  Detail & Related papers  (2020-06-24T17:53:53Z)
• Optimal Regularization Can Mitigate Double Descent [29.414119906479954]
  We study whether the double-descent phenomenon can be avoided by using optimal regularization. We demonstrate empirically that optimally-tuned $ell$ regularization can double descent for more general models, including neural networks.
  arXiv  Detail & Related papers  (2020-03-04T05:19:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.