How Sparsity Allocation Shapes Label-Free Post-Pruning Recoverability

• URL: http://arxiv.org/abs/2605.21972v1
• Date: Thu, 21 May 2026 04:03:24 GMT
• Title: How Sparsity Allocation Shapes Label-Free Post-Pruning Recoverability
• Abstract summary: Post-pruning repair methods can partially recover collapsed sparse models, but their effectiveness depends on the sparse model left by the upstream pruning allocation.<n>This paper studies how sparsity allocation shapes post-repair recoverability under a fixed activation-statistic repair backend.
• Score: 4.9112670395105305
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unstructured magnitude pruning at high sparsity can reduce neural network accuracy to near-random performance, while labeled retraining may be unavailable in practical deployment settings. Label-free post-pruning repair methods can partially recover collapsed sparse models, but their effectiveness depends on the sparse model left by the upstream pruning allocation. This paper studies how sparsity allocation shapes post-repair recoverability under a fixed activation-statistic repair backend. We compare ERK and LAMP allocations under the same label-free repair protocol across CIFAR-10, CIFAR-100, and Imagenette with ResNet-18, ResNet-34, and ResNet-50 at sparsities from 90% to 95.5%. The results show that allocation choice can substantially change post-repair accuracy at the same global sparsity, and that the preferred allocation varies with architecture, dataset difficulty, and sparsity level. We identify a repair-sensitive transition regime in which BatchNorm recalibration begins to fail, while activation-statistic repair still recovers nontrivial accuracy. Additional validation on ImageNet-100 and DenseNet-121 shows that the location and width of this recoverable regime depend on data scale and connectivity structure. These findings suggest that pruning allocation and post-pruning repair should be studied jointly, since the allocation determines how much activation signal remains available for label-free recovery.

Related papers

• Restore What Matters: Lessons from Joint Restoration and Recognition [72.91692240810536]
  Recognition pipelines typically adopt a restore-then-recognize workflow, yet decades of experience show that generating visually pleasing images seldom to improved recognition.<n>We propose a Joint Restoration-for-Recognition paradigm: restore only what downstream tasks truly require, with task signals dictating where, how much, and whether restoration is necessary.<n>We conclude that better-looking images are neither necessary nor sufficient; restoration modules must be task-driven, selective, and physically aware.
  arXiv  Detail & Related papers  (2026-09-12T08:08:08Z)
• TTSD-FAR: Test-Time Self-Distillation with Fisher-Anchored Restoration for Missing-Modality Emotion Recognition in LVLMs [83.78708832901194]
  Large video-language models (LVLMs) have shown remarkable performance on multimodal tasks like multimodal emotion recognition (ER) in the wild.<n>However, real-world deployment remains challenging: modalities may be missing or noisy at test time.<n>We propose a solution in which a frozen teacher guides an adaptive low-rank student via self-distillation.<n>Experiments on MELD, DFEW, and BAH under 0%-50% missing modalities show that this unified adaptation-restoration design consistently outperforms entropy-based adaptation, RAG, and perplexity-based generation.
  arXiv  Detail & Related papers  (2026-08-18T23:40:28Z)
• Bend the Basics: Degradation-Aware Deformable Tokenization for All-in-One Image Restoration [72.64498743287601]
  All-in-one image restoration seeks a single model that can recover images degraded by diverse and spatially non-uniform corruptions.<n>We present Flexible Image Transformer (FIT) that explicitly models degradation awareness across the entire pipeline.<n>FIT achieves state-of-the-art performance with 30.72 dB average PSNR on the five-degradation setting and 32.83 dB on the three-degradation setting, outperforming recent unified restoration methods by +0.5$sim$1.1 dB.
  arXiv  Detail & Related papers  (2026-08-07T05:44:31Z)
• Relative Repairability: A Calibration-Based Diagnostic for High-Sparsity Post-Pruning Allocation [12.288779306467768]
  Relative Repairability (RR) is a calibration based diagnostic that compares the raw activation distortion caused by layerwise pruning with the residual distortion left after channelwise variance matching repair.<n>RR is most useful near an architecture dependent recoverability transition, where standard structural or magnitude based allocation priors begin to lose reliability but post repair recovery has not yet fully collapsed.<n>These results suggest that high sparsity pruning should allocate not only retained weights, but also repairable damage.
  arXiv  Detail & Related papers  (2026-05-25T07:12:38Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• Adaptive Signal Resuscitation: Channel-wise Post-Pruning Repair for Sparse Vision Networks [8.5133415329194]
  One-shot magnitude pruning can cause severe accuracy collapse in the high-sparsity regime.<n>We propose Adaptive Signal Resuscitation (ASR) as a training-free channel-wise repair method.<n>ASR estimates a variance-matching correction for each output channel and stabilizes it with a data-driven shrinkage rule.
  arXiv  Detail & Related papers  (2026-05-20T17:19:01Z)
• DegBins: Degradation-Driven Binning for Depth Super-Resolution [57.61783461543096]
  DegBins is a novel DSR framework that leverages degradation-driven binning to adaptively enhance residual modeling.<n>DegBins consistently outperforms existing state-of-the-art methods in terms of accuracy, robustness, and generalization.
  arXiv  Detail & Related papers  (2026-05-10T16:17:11Z)
• Restoration-Aligned Generative Flow Models for Blind Motion Deblurring [57.425021485907756]
  We present DeFlow, a framework that resolves misalignment by reformulating the flow trajectory itself.<n>Under this formulation, the standard flow matching loss naturally takes the form of a residual loss.<n>Experiments on GoPro, HIDE, RealBlur, and RWBI demonstrate that DeFlow achieves strong restoration fidelity and perceptual realism.
  arXiv  Detail & Related papers  (2026-05-09T10:05:51Z)
• Exactness Matters for Physical Rule Enforcement [4.230349220081444]
  It remains unclear when stronger physical rule enforcement becomes reliable and when it becomes a source of distribution shift.<n>We study this question through operator exactness, meaning whether the repair map is the identity on the target manifold.<n> Controlled mismatch, screened cleanup, adaptive gating, and external-backbone checks show that the best approximate-regime operating point can be raw or near-identity.
  arXiv  Detail & Related papers  (2026-05-08T07:59:59Z)
• ANNEAL: Adapting LLM Agents via Governed Symbolic Patch Learning [12.44509691077682]
  We introduce ANNEAL, a neuro-symbolic agent that converts recurring failures into governed symbolic edits of a process knowledge graph.<n>Its core mechanism, Failure-Driven Knowledge Acquisition (FDKA), localizes the responsible operator, synthesizes a typed patch through constrained LLM generation, and validates the proposal.<n>Across four domains and 27 multi-seed runs, ANNEAL is the only evaluated system that commits persistent structural repairs.
  arXiv  Detail & Related papers  (2026-05-04T05:24:03Z)
• Variational Rectification Inference for Learning with Noisy Labels [74.85528327499662]
  We propose variational rectification inference (VRI) to formulate the adaptive rectification for loss functions.<n>VRI is constructed as a hierarchical Bayes by treating the rectifying vector as a latent variable.<n>By introducing the variational term in VRI, the conditional posterior is estimated accurately and avoids collapsing to a Dirac delta function.
  arXiv  Detail & Related papers  (2026-03-18T01:25:08Z)
• Rethinking Reconstruction Autoencoder-Based Out-of-Distribution Detection [0.0]
  Reconstruction autoencoder-based methods deal with the problem by using input reconstruction error as a metric of novelty vs. normality. We introduce semantic reconstruction, data certainty decomposition and normalized L2 distance to substantially improve original methods. Our method works without any additional data, hard-to-implement structure, time-consuming pipeline, and even harming the classification accuracy of known classes.
  arXiv  Detail & Related papers  (2022-03-04T09:04:55Z)
• Edge Rewiring Goes Neural: Boosting Network Resilience via Policy Gradient [62.660451283548724]
  ResiNet is a reinforcement learning framework to discover resilient network topologies against various disasters and attacks. We show that ResiNet achieves a near-optimal resilience gain on multiple graphs while balancing the utility, with a large margin compared to existing approaches.
  arXiv  Detail & Related papers  (2021-10-18T06:14:28Z)
• Structured and Localized Image Restoration [141.75042935077465]
  We present a novel approach to image restoration that leverages ideas from localized structured prediction and non-linear multi-task learning. We derive the corresponding algorithms for energies based on the mean-squared and Euclidean norm errors.
  arXiv  Detail & Related papers  (2020-06-16T15:43:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.