Compression Aware Certified Training

• URL: http://arxiv.org/abs/2506.11992v1
• Date: Fri, 13 Jun 2025 17:48:50 GMT
• Title: Compression Aware Certified Training
• Abstract summary: Existing methods treat compression and certified robustness as separate goals, compromising either efficiency or safety.<n>We propose CACTUS, a general framework for unifying these objectives during training.<n>We apply CACTUS for both pruning and quantization and show that it effectively trains models which can be efficiently compressed.
• Score: 4.456428506059651
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep neural networks deployed in safety-critical, resource-constrained environments must balance efficiency and robustness. Existing methods treat compression and certified robustness as separate goals, compromising either efficiency or safety. We propose CACTUS (Compression Aware Certified Training Using network Sets), a general framework for unifying these objectives during training. CACTUS models maintain high certified accuracy even when compressed. We apply CACTUS for both pruning and quantization and show that it effectively trains models which can be efficiently compressed while maintaining high accuracy and certifiable robustness. CACTUS achieves state-of-the-art accuracy and certified performance for both pruning and quantization on a variety of datasets and input specifications.

Related papers

• Improving Certified Robustness via Adversarial Distillation [2.969291853301593]
  We introduce AD-CERT, a certified training objective that combines adversarial distillation with an IBP upper bound.<n>We show that AD-CERT achieves state-of-the-art certified performance on several benchmarks.
  arXiv  Detail & Related papers  (2026-06-30T13:31:35Z)
• Certified Robustness from Approximate Gaussian Mixture Structures in Pretrained Latent Spaces [42.97530488302465]
  Deep learning models are vulnerable to adversarial perturbations.<n> Empirical defenses can achieve strong robustness in practice, but lack formal guarantees.<n>We propose a framework for designing certifiably robust classifiers that leverage latent structure in data representations.
  arXiv  Detail & Related papers  (2026-05-25T02:12:58Z)
• Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration [72.0672328514289]
  Multimodal learning often grapples with the challenge of low-quality data.<n>We propose a unified framework, termed Conformal Predictive Self-Calibration.<n>Our framework consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-05T14:48:52Z)
• Learning Better Certified Models from Empirically-Robust Teachers [53.51898477987441]
  Adrial training attains strong empirical robustness to specific adversarial attacks by training on concrete adversarial perturbations.<n>It produces neural networks that are not amenable to strong robustness certificates through neural network verification.<n>Earlier certified training schemes directly train on bounds from network relaxations to obtain models that are certifiably robust, but display sub-par standard performance.
  arXiv  Detail & Related papers  (2026-02-02T16:30:53Z)
• Enhancing Trustworthiness with Mixed Precision: Benchmarks, Opportunities, and Challenges [12.438306093697]
  Large language models (LLMs) have shown promising performance across various tasks.<n>LLMs' autoregressive decoding process poses significant challenges for efficient deployment on existing AI hardware.
  arXiv  Detail & Related papers  (2025-11-27T14:17:43Z)
• Efficient Token Compression for Vision Transformer with Spatial Information Preserved [59.79302182800274]
  Token compression is essential for reducing the computational and memory requirements of transformer models.<n>We propose an efficient and hardware-compatible token compression method called Prune and Merge.
  arXiv  Detail & Related papers  (2025-03-30T14:23:18Z)
• Data-Aware Training Quality Monitoring and Certification for Reliable Deep Learning [13.846014191157405]
  We introduce YES training bounds, a novel framework for real-time, data-aware certification and monitoring of neural network training. We show that YES bounds offer insights beyond conventional local optimization perspectives, such as identifying when training losses plateau in suboptimal regions. We offer a powerful tool for real-time evaluation, setting a new standard for training quality assurance in deep learning.
  arXiv  Detail & Related papers  (2024-10-14T18:13:22Z)
• Cross-Input Certified Training for Universal Perturbations [4.456428506059651]
  Current certified training methods train models robust to single-input perturbations but achieve suboptimal clean and UAP accuracy. We propose a novel method, CITRUS, for certified training of networks robust against UAP attackers. We show in an extensive evaluation across different datasets, architectures, and perturbation magnitudes that our method outperforms traditional certified training methods on standard accuracy (up to 10.3%) and achieves SOTA performance on the more practical certified UAP accuracy metric.
  arXiv  Detail & Related papers  (2024-05-15T08:33:41Z)
• PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation [61.57833648734164]
  We propose a novel Parallel Yielding Re-Activation (PYRA) method for training-inference efficient task adaptation. PYRA outperforms all competing methods under both low compression rate and high compression rate.
  arXiv  Detail & Related papers  (2024-03-14T09:06:49Z)
• Adaptive Certified Training: Towards Better Accuracy-Robustness Tradeoffs [17.46692880231195]
  We propose a novel certified training method based on a key insight that training with adaptive certified radii helps to improve the accuracy and robustness of the model. We demonstrate the effectiveness of the proposed method on MNIST, CIFAR-10, and TinyImageNet datasets.
  arXiv  Detail & Related papers  (2023-07-24T18:59:46Z)
• Augment then Smooth: Reconciling Differential Privacy with Certified Robustness [32.49465965847842]
  We show that standard differentially private model training is insufficient for providing strong certified robustness guarantees.<n>We present DP-CERT, a simple and effective method that achieves both privacy and robustness guarantees simultaneously.
  arXiv  Detail & Related papers  (2023-06-14T17:52:02Z)
• CrAFT: Compression-Aware Fine-Tuning for Efficient Visual Task Adaptation [3.043665249713003]
  Post-training compression techniques such as pruning and quantization can help lower deployment costs. We propose CrAFT, a simple fine-tuning framework that enables effective post-training network compression. The CrAFT approach adds negligible training overhead as fine-tuning is done in under a couple of minutes or hours with a single GPU.
  arXiv  Detail & Related papers  (2023-05-08T07:51:40Z)
• LCS: Learning Compressible Subspaces for Adaptive Network Compression at Inference Time [57.52251547365967]
  We propose a method for training a "compressible subspace" of neural networks that contains a fine-grained spectrum of models. We present results for achieving arbitrarily fine-grained accuracy-efficiency trade-offs at inference time for structured and unstructured sparsity. Our algorithm extends to quantization at variable bit widths, achieving accuracy on par with individually trained networks.
  arXiv  Detail & Related papers  (2021-10-08T17:03:34Z)
• Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression [49.288196234823005]
  We propose two new metrics, label loyalty and probability loyalty that measure how closely a compressed model mimics the original model. We also explore the effect of compression with regard to robustness under adversarial attacks.
  arXiv  Detail & Related papers  (2021-09-07T17:55:47Z)
• AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks [78.62086125399831]
  We present a general approach called Alternating Compressed/DeCompressed (AC/DC) training of deep neural networks (DNNs) AC/DC outperforms existing sparse training methods in accuracy at similar computational budgets. An important property of AC/DC is that it allows co-training of dense and sparse models, yielding accurate sparse-dense model pairs at the end of the training process.
  arXiv  Detail & Related papers  (2021-06-23T13:23:00Z)
• Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Free [115.81899803240758]
  Adversarial training and its many variants substantially improve deep network robustness, yet at the cost of compromising standard accuracy. This paper asks how to quickly calibrate a trained model in-situ, to examine the achievable trade-offs between its standard and robust accuracies. Our proposed framework, Once-for-all Adversarial Training (OAT), is built on an innovative model-conditional training framework.
  arXiv  Detail & Related papers  (2020-10-22T16:06:34Z)
• Regularized Training and Tight Certification for Randomized Smoothed Classifier with Provable Robustness [15.38718018477333]
  We derive a new regularized risk, in which the regularizer can adaptively encourage the accuracy and robustness of the smoothed counterpart. We also design a new certification algorithm, which can leverage the regularization effect to provide tighter robustness lower bound that holds with high probability.
  arXiv  Detail & Related papers  (2020-02-17T20:54:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.