Double-Scoring: Reliable Extraction of Strong Lottery Tickets

• URL: http://arxiv.org/abs/2607.20555v1
• Date: Wed, 15 Jul 2026 20:20:43 GMT
• Title: Double-Scoring: Reliable Extraction of Strong Lottery Tickets
• Abstract summary: Lottery ticket hypothesis proposes that large random neural networks contain sparseworks that can match the performance of dense models after comparable training.<n>Existing theory establishes that such strong lottery tickets exist, but reliable extraction remains difficult.<n>We revisit edge-popup, a frozen-weight score-training method for extracting strong tickets, and identify layerwise sparsity selection as a central bottleneck.<n>We show that fixed-density masking in an augmented score space preserves access to all original-coordinate masks, and we show that the resulting method can be interpreted as edge-popup on a zero-augmented network.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The lottery ticket hypothesis proposes that large random neural networks contain sparse subnetworks that can match the performance of dense models after comparable training. A stronger version asserts that sufficiently overparameterized random networks contain subnetworks that are already accurate before any weight training. Existing theory establishes that such strong lottery tickets exist, but reliable extraction remains difficult. We revisit edge-popup, a frozen-weight score-training method for extracting strong tickets, and identify layerwise sparsity selection as a central bottleneck. We introduce double-scoring, an augmented score-space parameterization that replaces a layerwise sparsity search with optimization over enlarged score tensors. We prove that fixed-density masking in an augmented score space preserves access to all original-coordinate masks, and we show that the resulting method can be interpreted as edge-popup on a zero-augmented network. In controlled experiments, double-scoring substantially improves strong-ticket extraction over fixed-density edge-popup and pruning-at-initialization baselines, improves on the performance of rewound sparse-training topologies, and exhibits markedly lower sensitivity to sparsity hyperparameters. Ablations show that the gain is not merely due to additional trainable score parameters, but is tied to the augmented score-space competition that induces the effective original sparsity.

Related papers

• Uncovering a Winning Lottery Ticket with Continuously Relaxed Bernoulli Gates [13.591018807414484]
  Strong Lottery Ticket (SLT) hypothesis suggests randomly networks contain sparseworks achieving competitive accuracy without weight training.<n>We propose using continuously relaxed Bernoulligating to discover SLTs through fully differentiable, end-to-end optimization.<n> Experiments across fully connected networks, CNNs (ResNet, Wide-ResNet), and Vision Transformers (ViT, Swin-T) demonstrate up to 90% sparsity with minimal accuracy loss.
  arXiv  Detail & Related papers  (2026-03-09T20:33:16Z)
• Find A Winning Sign: Sign Is All We Need to Win the Lottery [52.63674911541416]
  We show that a sparse network trained by an existing IP method can retain its basin of attraction if its parameter signs and normalization layer parameters are preserved.<n>To take a step closer to finding a winning ticket, we alleviate the reliance on normalization layer parameters by preventing high error barriers along the linear path between the sparse network trained by our method and its counterpart with normalization layer parameters.
  arXiv  Detail & Related papers  (2025-04-07T09:30:38Z)
• Playing the Lottery With Concave Regularizers for Sparse Trainable Neural Networks [10.48836159692231]
  We propose a novel class of methods to play the lottery.<n>The key point is the use of concave regularization to promote the sparsity of a relaxed binary mask.<n>We show that the proposed method can improve the performance of state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2025-01-19T18:05:13Z)
• Binarizing Sparse Convolutional Networks for Efficient Point Cloud Analysis [93.55896765176414]
  We propose binary sparse convolutional networks called BSC-Net for efficient point cloud analysis. We employ the differentiable search strategies to discover the optimal opsitions for active site matching in the shifted sparse convolution. Our BSC-Net achieves significant improvement upon our srtong baseline and outperforms the state-of-the-art network binarization methods.
  arXiv  Detail & Related papers  (2023-03-27T13:47:06Z)
• Why Random Pruning Is All We Need to Start Sparse [7.648170881733381]
  Random masks define surprisingly effective sparse neural network models. We show that sparser networks can compete with dense architectures and state-of-the-art lottery ticket pruning algorithms.
  arXiv  Detail & Related papers  (2022-10-05T17:34:04Z)
• Dual Lottery Ticket Hypothesis [71.95937879869334]
  Lottery Ticket Hypothesis (LTH) provides a novel view to investigate sparse network training and maintain its capacity. In this work, we regard the winning ticket from LTH as the subnetwork which is in trainable condition and its performance as our benchmark. We propose a simple sparse network training strategy, Random Sparse Network Transformation (RST), to substantiate our DLTH.
  arXiv  Detail & Related papers  (2022-03-08T18:06:26Z)
• The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training [111.15069968583042]
  Random pruning is arguably the most naive way to attain sparsity in neural networks, but has been deemed uncompetitive by either post-training pruning or sparse training. We empirically demonstrate that sparsely training a randomly pruned network from scratch can match the performance of its dense equivalent. Our results strongly suggest there is larger-than-expected room for sparse training at scale, and the benefits of sparsity might be more universal beyond carefully designed pruning.
  arXiv  Detail & Related papers  (2022-02-05T21:19:41Z)
• Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks [79.74580058178594]
  We analyze the performance of training a pruned neural network by analyzing the geometric structure of the objective function. We show that the convex region near a desirable model with guaranteed generalization enlarges as the neural network model is pruned.
  arXiv  Detail & Related papers  (2021-10-12T01:11:07Z)
• FreeTickets: Accurate, Robust and Efficient Deep Ensemble by Training with Dynamic Sparsity [74.58777701536668]
  We introduce the FreeTickets concept, which can boost the performance of sparse convolutional neural networks over their dense network equivalents by a large margin. We propose two novel efficient ensemble methods with dynamic sparsity, which yield in one shot many diverse and accurate tickets "for free" during the sparse training process.
  arXiv  Detail & Related papers  (2021-06-28T10:48:20Z)
• ESPN: Extremely Sparse Pruned Networks [50.436905934791035]
  We show that a simple iterative mask discovery method can achieve state-of-the-art compression of very deep networks. Our algorithm represents a hybrid approach between single shot network pruning methods and Lottery-Ticket type approaches.
  arXiv  Detail & Related papers  (2020-06-28T23:09:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.