CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning

• URL: http://arxiv.org/abs/2607.17568v1
• Date: Mon, 20 Jul 2026 05:27:57 GMT
• Title: CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning
• Abstract summary: CoCurve is a calibration-only, fine-tuning-free method that prunes attention and FFN units jointly.<n>It reduces to one budgeted quadratic program, solved in a single shot under a shared attention--FFN budget, with no labels, fine-tuning, or recovery.
• Score: 47.406985340436925
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Structured pruning compresses large language models (LLMs) by removing whole computational units, such as attention heads and feed-forward (FFN) channel groups. Most training-free methods, however, rank these units independently, implicitly treating the loss from pruning a set as the sum of its individual losses. This view fails for Transformers, whose sublayers are coupled through a shared residual stream. Two individually weak units can thus be jointly indispensable, yet independent scoring is blind to such dependence and removes them together. We introduce CoCurve (Cross-Module Co-Pruning Curvature), a calibration-only, fine-tuning-free method that prunes attention and FFN units jointly. A second-order Taylor expansion of the token-level KL between the frozen model and its masked copy yields a single Fisher matrix whose diagonal is classical node saliency and whose off-diagonal entries are co-pruning curvature edges: the extra damage of removing two units together. Under a single-ablation additivity approximation this matrix reduces to a Gram product of single-unit ablation features, so the full M x M interaction is recovered from M forward passes, with no pairwise sweeps or gradients. Pruning then reduces to one budgeted quadratic program, solved in a single shot under a shared attention--FFN budget, with no labels, fine-tuning, or recovery.

Related papers

• Cryptanalytic Extraction of Isolated Bias-Free GLU Feed-Forward Blocks by Antipodal Separation [6.560534698339666]
  Cryptanalytic extraction has been demonstrated for ReLU networks, for networks using componentwise activations such as GELU or SiLU, and for a Transformer's final projection matrix.<n>These methods do not recover the bias-free Gated Linear Unit feed-forward blocks used in many modern language models.<n>We give a constructive, multi-stage forward-query recovery primitive for isolated bias-free GLU blocks.
  arXiv  Detail & Related papers  (2026-08-06T22:41:40Z)
• Certified boundary-magic witness for state-dependent proto-area in a holographic code [0.0]
  We study a four-qubit product-EPR holographic code whose reconstructing region contains matter and one leg of a geometry bond.<n>We show that only matter-controlled bond motion produces a leading quadratic logical-state dependence of the boundary entropy.
  arXiv  Detail & Related papers  (2026-07-12T09:16:46Z)
• Closed-Form Spectral Regularization for Multi-Task Model Merging [96.82449201305234]
  Model merging combines several independently fine-tuned experts into a single multi-task model without any training data.<n>State-of-the-art merging methods formulate merging as a layer-wise interference problem.<n>We propose SWUDI, a closed-form method that combines a soft exponential filter, which matches the gradient-flow trajectory of iterative descent.
  arXiv  Detail & Related papers  (2026-06-05T14:00:47Z)
• LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models [56.999481798138625]
  LimiX-2M is a 2M- parameter model that outperforms larger TabPFN-v2 and TabICL baselines on widely used benchmarks.<n>We present a unified tokenize-and-route framework for strong Tabular foundation models (TFMs)<n>Results highlight value-aware tokenization and readout-aligned routing as key levers for improving the accuracy--efficiency trade-off in TFMs.
  arXiv  Detail & Related papers  (2026-06-03T06:07:33Z)
• GeoFlowVLM: Geometry-Aware Joint Uncertainty for Frozen Vision-Language Embedding [3.0708725114491293]
  We propose textbfGeoFlowVLM as a post-hoc adapter that learns the joint distribution of paired $ell$-normalised dual-encoder VLM embeddings.<n>A consistency result shows that, in the population limit, the trained network exposes the joint flow and both cross-modal conditional flows.<n>We derive two quantities from this single model: a conditional retrieval entropy that quantifies aleatoric ambiguity with a decision-theoretic interpretation via a Fano-type bound, and a marginaltypicality score justified by an exact chain-rule decomposition of the joint NLL
  arXiv  Detail & Related papers  (2026-05-13T11:12:18Z)
• Operator Spectroscopy of Trained Lattice Samplers [0.0]
  We analyze the trained field-space function itself.<n>We project these functions onto operator bases fixed before the fit.<n>The operator basis is model- and symmetry-dependent, but the test is common.
  arXiv  Detail & Related papers  (2026-05-11T20:06:15Z)
• DDCL: Deep Dual Competitive Learning: A Differentiable End-to-End Framework for Unsupervised Prototype-Based Representation Learning [0.0]
  A persistent structural weakness in deep clustering is the disconnect between feature learning and cluster assignment.<n>This paper introduces Deep Dual Competitive Learning, the first fully differentiable end-to-end framework for unsupervised representation learning.
  arXiv  Detail & Related papers  (2026-04-02T08:02:27Z)
• Don't Be Greedy, Just Relax! Pruning LLMs via Frank-Wolfe [61.68406997155879]
  State-of-the-art Large Language Model (LLM) pruning methods operate layer-wise, minimizing the per-layer pruning error on a small dataset to avoid full retraining.<n>Existing methods hence rely on greedy convexs that ignore the weight interactions in the pruning objective.<n>Our method drastically reduces the per-layer pruning error, outperforms strong baselines on state-of-the-art GPT architectures, and remains memory-efficient.
  arXiv  Detail & Related papers  (2025-10-15T16:13:44Z)
• Constrained Edge AI Deployment: Fine-Tuning vs Distillation for LLM Compression [1.85373927927491]
  Modern models are often compressed via a combination of structured pruning and re-training to meet the strict compute, memory, and connectivity constraints of edge deployments.<n>Our focus is not on achieving maximal compression, but on isolating the impact of the re-training loss function.<n>We evaluate both pipelines on the OLMo2- 7B-SFT model for CommonsenseQA suitable for intermittent or denied connectivity scenarios typical of edge networks.
  arXiv  Detail & Related papers  (2025-05-13T19:06:32Z)
• Lifting the Convex Conjugate in Lagrangian Relaxations: A Tractable Approach for Continuous Markov Random Fields [53.31927549039624]
  We show that a piecewise discretization preserves better contrast from existing discretization problems. We apply this theory to the problem of matching two images.
  arXiv  Detail & Related papers  (2021-07-13T12:31:06Z)
• Competitive Mirror Descent [67.31015611281225]
  Constrained competitive optimization involves multiple agents trying to minimize conflicting objectives, subject to constraints. We propose competitive mirror descent (CMD): a general method for solving such problems based on first order information. As a special case we obtain a novel competitive multiplicative weights algorithm for problems on the positive cone.
  arXiv  Detail & Related papers  (2020-06-17T22:11:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.