Slimmable ConvNeXt: Width-Adaptive Inference for Efficient Multi-Device Deployment

• URL: http://arxiv.org/abs/2605.22677v1
• Date: Thu, 21 May 2026 16:20:18 GMT
• Title: Slimmable ConvNeXt: Width-Adaptive Inference for Efficient Multi-Device Deployment
• Abstract summary: We present Slimmable ConvNeXt, which shows that ConvNeXt's modern design, specifically LayerNorm and inverted bottlenecks, makes it particularly suited for channel-width slimming.<n>On ImageNet-1k, Slimmable ConvNeXt-T with 3works achieves 80.8% top-1 accuracy at 4.5 GMACs and 77.4% at 1.2 GMACs, trained from scratch for 600 epochs.
• Score: 4.976815699476327
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deploying vision models across devices with varying resource constraints, or even on a single device where available compute fluctuates due to battery state, thermal throttling, or latency deadlines, typically requires training and maintaining separate models. Width-adaptive inference addresses this by training a single set of shared weights containing multiple nested subnetworks of increasing capacity, but prior CNN-based approaches required switchable batch normalization, while recent scalable methods have focused on Vision Transformers. We present Slimmable ConvNeXt, which shows that ConvNeXt's modern design, specifically LayerNorm and inverted bottlenecks, makes it particularly suited for channel-width slimming, eliminating the normalization overhead of classical slimmable networks and producing a simpler training pipeline than both prior CNN and ViT approaches. On ImageNet-1k, Slimmable ConvNeXt-T with 3 subnetworks achieves 80.8% top-1 accuracy at 4.5 GMACs and 77.4% at 1.2 GMACs, trained from scratch for 600 epochs. At comparable compute, this exceeds HydraViT's 6-head subnetwork (78.4% at 4.6 GMACs) by 2.4 percentage points and its 3-head configuration (73.0% at 1.3 GMACs) by 4.4 percentage points, while also outperforming MatFormer-S (78.6%) and SortedNet-S (78.2%) at the same GMACs. Scaling to Slimmable ConvNeXt-B further improves maximum accuracy to 82.8% at 15.35 GMACs.

Related papers

• WaVeFuse: Regime-Adaptive Equity Index Forecasting via Channel-Wise Wavelet Denoising and Vertical Attention Fusion [0.0]
  WaVeFuse is a hybrid Deep Learning for equity forecasting framework.<n>It addresses propagation of technical indicators (TIs), channel-indiscriminate decomposition, and static multi-branch fusion.<n>WaVeFuse delivers a computationally efficient, regime-robust framework suitable for research and decision-support deployment.
  arXiv  Detail & Related papers  (2026-09-13T18:50:33Z)
• Do Newer Lightweight CNNs Perform Better Under Resource Constraints? A Controlled Multigenerational Study of Architecture, Initialization, Training Budget, and Efficiency [0.0]
  This study compares nine lightweight CNN model packages across CIFAR-10, CIFAR-100, and Tiny ImageNet.<n>We report top-1 accuracy, macro F1, top-5 accuracy, parameter count, FP32 storage, GMACs, batch-size-1 latency on an NVIDIA L4 and AMD 5 5500U CPU.
  arXiv  Detail & Related papers  (2026-07-02T10:14:07Z)
• CLEAR-MoE: Shared-Basis Expert Extraction from Frozen Vision Transformers via Calibration-Driven Layer Selection [0.0]
  CLEAR-MoE is a post-training pipeline that converts a frozen pretrained Vision Transformer (ViT) into a sparse Mixture-of-Experts (MoE) model without updating backbone weights.<n>On a GTX 960, routing and scatter-gather overhead make the CLEAR-MoE FFN 1.3-1.7x slower than the dense routers.
  arXiv  Detail & Related papers  (2026-06-26T18:12:03Z)
• FeatCal: Feature Calibration for Post-Merging Models [37.524690106426284]
  Model merging combines task experts into one model and avoids joint training, retraining, or deploying many expert models.<n>We study this performance gap through feature drift, the difference between features produced by the merged model and by the expert on the same input.<n>This view motivates FeatCal, which uses a small calibration set to calibrate the merged model weights layer by layer in forward order.
  arXiv  Detail & Related papers  (2026-05-13T05:35:03Z)
• SLNet: A Super-Lightweight Geometry-Adaptive Network for 3D Point Cloud Recognition [1.499944454332829]
  We present SLNet, a lightweight backbone for 3D point cloud recognition.<n>The model is built on two simple ideas: NAPE (Nonparametric Adaptive Point Embedding) and GMU (Geometric Modulation Unit)
  arXiv  Detail & Related papers  (2026-03-08T04:07:42Z)
• Progressive Supernet Training for Efficient Visual Autoregressive Modeling [56.15415456746672]
  We propose a training strategy that breaks through the frontier of generation quality for both paradigms and the full network.<n>Experiments on ImageNet demonstrate that, compared to the pretrained VAR-d30, VARiant-d16 and VARiant-d8 achieve nearly equivalent quality.<n> VARiant-d2 achieves 3.5 times speedup and 80% memory reduction at moderate quality cost.
  arXiv  Detail & Related papers  (2025-11-20T16:59:24Z)
• Time Frequency Analysis of EMG Signal for Gesture Recognition using Fine grained Features [3.9440964696313485]
  This paper proposes a novel approach to EMG-based hand gesture recognition that uses fine-grained classification.<n> XMANet unifies low-level local and high level semantic cues through cross layer mutual attention among shallow to deep CNN experts.
  arXiv  Detail & Related papers  (2025-04-20T18:51:10Z)
• ParFormer: A Vision Transformer with Parallel Mixer and Sparse Channel Attention Patch Embedding [9.144813021145039]
  This paper introduces ParFormer, a vision transformer that incorporates a Parallel Mixer and a Sparse Channel Attention Patch Embedding (SCAPE) ParFormer improves feature extraction by combining convolutional and attention mechanisms. For edge device deployment, ParFormer-T excels with a throughput of 278.1 images/sec, which is 1.38 $times$ higher than EdgeNeXt-S. The larger variant, ParFormer-L, reaches 83.5% Top-1 accuracy, offering a balanced trade-off between accuracy and efficiency.
  arXiv  Detail & Related papers  (2024-03-22T07:32:21Z)
• Quantized Neural Networks for Low-Precision Accumulation with Guaranteed Overflow Avoidance [68.8204255655161]
  We introduce a quantization-aware training algorithm that guarantees avoiding numerical overflow when reducing the precision of accumulators during inference. We evaluate our algorithm across multiple quantized models that we train for different tasks, showing that our approach can reduce the precision of accumulators while maintaining model accuracy with respect to a floating-point baseline.
  arXiv  Detail & Related papers  (2023-01-31T02:46:57Z)
• EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications [68.35683849098105]
  We introduce split depth-wise transpose attention (SDTA) encoder that splits input tensors into multiple channel groups. Our EdgeNeXt model with 1.3M parameters achieves 71.2% top-1 accuracy on ImageNet-1K. Our EdgeNeXt model with 5.6M parameters achieves 79.4% top-1 accuracy on ImageNet-1K.
  arXiv  Detail & Related papers  (2022-06-21T17:59:56Z)
• DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers [105.74546828182834]
  We show a hardware-efficient dynamic inference regime, named dynamic weight slicing, which adaptively slice a part of network parameters for inputs with diverse difficulty levels. We present dynamic slimmable network (DS-Net) and dynamic slice-able network (DS-Net++) by input-dependently adjusting filter numbers of CNNs and multiple dimensions in both CNNs and transformers.
  arXiv  Detail & Related papers  (2021-09-21T09:57:21Z)
• Greedy Network Enlarging [53.319011626986004]
  We propose a greedy network enlarging method based on the reallocation of computations. With step-by-step modifying the computations on different stages, the enlarged network will be equipped with optimal allocation and utilization of MACs. With application of our method on GhostNet, we achieve state-of-the-art 80.9% and 84.3% ImageNet top-1 accuracies.
  arXiv  Detail & Related papers  (2021-07-31T08:36:30Z)
• Non-Parametric Adaptive Network Pruning [125.4414216272874]
  We introduce non-parametric modeling to simplify the algorithm design. Inspired by the face recognition community, we use a message passing algorithm to obtain an adaptive number of exemplars. EPruner breaks the dependency on the training data in determining the "important" filters.
  arXiv  Detail & Related papers  (2021-01-20T06:18:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.