Enhancing Variable Selection in Large-scale Logistic Regression: Leveraging Manual Labeling with Beneficial Noise

• URL: http://arxiv.org/abs/2504.16585v1
• Date: Wed, 23 Apr 2025 10:05:54 GMT
• Title: Enhancing Variable Selection in Large-scale Logistic Regression: Leveraging Manual Labeling with Beneficial Noise
• Authors: Xiaofei Wu, Rongmei Liang,
• Abstract summary: In large-scale supervised learning, penalized logistic regression (PLR) effectively addresses the overfitting problem by introducing regularization terms.<n>This paper theoretically demonstrates that label noise stemming from manual labeling, which is solely related to classification difficulty, represents a type of beneficial noise for variable selection in PLR.<n> Experimental results indicate that, as compared with traditional variable selection classification techniques, the PLR with manually-labeled noisy data achieves higher estimation and classification accuracy across multiple large-scale datasets.
• Score: 1.1477123412184609
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In large-scale supervised learning, penalized logistic regression (PLR) effectively addresses the overfitting problem by introducing regularization terms yet its performance still depends on efficient variable selection strategies. This paper theoretically demonstrates that label noise stemming from manual labeling, which is solely related to classification difficulty, represents a type of beneficial noise for variable selection in PLR. This benefit is reflected in a more accurate estimation of the selected non-zero coefficients when compared with the case where only truth labels are used. Under large-scale settings, the sample size for PLR can become very large, making it infeasible to store on a single machine. In such cases, distributed computing methods are required to handle PLR model with manual labeling. This paper presents a partition-insensitive parallel algorithm founded on the ADMM (alternating direction method of multipliers) algorithm to address PLR by incorporating manual labeling. The partition insensitivity of the proposed algorithm refers to the fact that the solutions obtained by the algorithm will not change with the distributed storage of data. In addition, the algorithm has global convergence and a sublinear convergence rate. Experimental results indicate that, as compared with traditional variable selection classification techniques, the PLR with manually-labeled noisy data achieves higher estimation and classification accuracy across multiple large-scale datasets.

Related papers

• Pseudo-label Refinement for Improving Self-Supervised Learning Systems [22.276126184466207]
  Self-supervised learning systems use clustering-based pseudo-labels to provide supervision without the need for human annotations. The noise in these pseudo-labels caused by the clustering methods poses a challenge to the learning process leading to degraded performance. We propose a pseudo-label refinement algorithm to address this issue.
  arXiv  Detail & Related papers  (2024-10-18T07:47:59Z)
• Inaccurate Label Distribution Learning with Dependency Noise [52.08553913094809]
  We introduce the Dependent Noise-based Inaccurate Label Distribution Learning (DN-ILDL) framework to tackle the challenges posed by noise in label distribution learning. We show that DN-ILDL effectively addresses the ILDL problem and outperforms existing LDL methods.
  arXiv  Detail & Related papers  (2024-05-26T07:58:07Z)
• All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation [67.30502812804271]
  Pseudo-labels are widely employed in weakly supervised 3D segmentation tasks where only sparse ground-truth labels are available for learning. We propose a novel learning strategy to regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions.
  arXiv  Detail & Related papers  (2023-05-25T08:19:31Z)
• Leveraging Instance Features for Label Aggregation in Programmatic Weak Supervision [75.1860418333995]
  Programmatic Weak Supervision (PWS) has emerged as a widespread paradigm to synthesize training labels efficiently. The core component of PWS is the label model, which infers true labels by aggregating the outputs of multiple noisy supervision sources as labeling functions. Existing statistical label models typically rely only on the outputs of LF, ignoring the instance features when modeling the underlying generative process.
  arXiv  Detail & Related papers  (2022-10-06T07:28:53Z)
• Partial sequence labeling with structured Gaussian Processes [8.239028141030621]
  We propose structured Gaussian Processes for partial sequence labeling. It encodes uncertainty in the prediction and does not need extra effort for model selection and hyper parameter learning. It is evaluated on several sequence labeling tasks and the experimental results show the effectiveness of the proposed model.
  arXiv  Detail & Related papers  (2022-09-20T00:56:49Z)
• Optimizing Diffusion Rate and Label Reliability in a Graph-Based Semi-supervised Classifier [2.4366811507669124]
  The Local and Global Consistency (LGC) algorithm is one of the most well-known graph-based semi-supervised (GSSL) classifiers. We discuss how removing the self-influence of a labeled instance may be beneficial, and how it relates to leave-one-out error. Within this framework, we propose methods to estimate label reliability and diffusion rate.
  arXiv  Detail & Related papers  (2022-01-10T16:58:52Z)
• Label Ranking through Nonparametric Regression [5.994412766684843]
  Label Ranking corresponds to the problem of learning a hypothesis that maps features to rankings over a finite set of labels. We introduce a generative model for Label Ranking, in noiseless and noisy nonparametric regression settings. We complement our theoretical contributions with experiments, aiming to understand how the input regression noise affects the observed output.
  arXiv  Detail & Related papers  (2021-11-04T10:59:46Z)
• Attentional-Biased Stochastic Gradient Descent [74.49926199036481]
  We present a provable method (named ABSGD) for addressing the data imbalance or label noise problem in deep learning. Our method is a simple modification to momentum SGD where we assign an individual importance weight to each sample in the mini-batch. ABSGD is flexible enough to combine with other robust losses without any additional cost.
  arXiv  Detail & Related papers  (2020-12-13T03:41:52Z)
• Delving Deep into Label Smoothing [112.24527926373084]
  Label smoothing is an effective regularization tool for deep neural networks (DNNs) We present an Online Label Smoothing (OLS) strategy, which generates soft labels based on the statistics of the model prediction for the target category.
  arXiv  Detail & Related papers  (2020-11-25T08:03:11Z)
• Evolving Multi-label Classification Rules by Exploiting High-order Label Correlation [2.9822184411723645]
  In multi-label classification tasks, each problem instance is associated with multiple classes simultaneously. The correlation between labels can be exploited at different levels such as capturing the pair-wise correlation or exploiting the higher-order correlations. This paper aims at exploiting the high-order label correlation within subsets of labels using a supervised learning classifier system.
  arXiv  Detail & Related papers  (2020-07-22T18:13:12Z)
• Progressive Identification of True Labels for Partial-Label Learning [112.94467491335611]
  Partial-label learning (PLL) is a typical weakly supervised learning problem, where each training instance is equipped with a set of candidate labels among which only one is the true label. Most existing methods elaborately designed as constrained optimizations that must be solved in specific manners, making their computational complexity a bottleneck for scaling up to big data. This paper proposes a novel framework of classifier with flexibility on the model and optimization algorithm.
  arXiv  Detail & Related papers  (2020-02-19T08:35:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.