FastBoost: Progressive Attention with Dynamic Scaling for Efficient Deep Learning

• URL: http://arxiv.org/abs/2511.01026v1
• Date: Sun, 02 Nov 2025 17:51:36 GMT
• Title: FastBoost: Progressive Attention with Dynamic Scaling for Efficient Deep Learning
• Authors: JunXi Yuan,
• Abstract summary: We present FastBoost, a parameter-efficient neural architecture that achieves state-of-the-art performance on CIFAR benchmarks.<n>Our design establishes new efficiency frontiers with: CIFAR-10: 95.57% accuracy (0.85M parameters) and 93.80% (0.37M parameters)<n>By integrating DSPA with enhanced MBConv blocks, FastBoost achieves a 2.1 times parameter reduction over MobileNetV3 while improving accuracy by +3.2 percentage points on CIFAR-10.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present FastBoost, a parameter-efficient neural architecture that achieves state-of-the-art performance on CIFAR benchmarks through a novel Dynamically Scaled Progressive Attention (DSPA) mechanism. Our design establishes new efficiency frontiers with: CIFAR-10: 95.57% accuracy (0.85M parameters) and 93.80% (0.37M parameters) CIFAR-100: 81.37% accuracy (0.92M parameters) and 74.85% (0.44M parameters) The breakthrough stems from three fundamental innovations in DSPA: (1) Adaptive Fusion: Learnt channel-spatial attention blending with dynamic weights. (2) Phase Scaling: Training-stage-aware intensity modulation (from 0.5 to 1.0). (3) Residual Adaptation: Self-optimized skip connections (gamma from 0.5 to 0.72). By integrating DSPA with enhanced MBConv blocks, FastBoost achieves a 2.1 times parameter reduction over MobileNetV3 while improving accuracy by +3.2 percentage points on CIFAR-10. The architecture features dual attention pathways with real-time weight adjustment, cascaded refinement layers (increasing gradient flow by 12.7%), and a hardware-friendly design (0.28G FLOPs). This co-optimization of dynamic attention and efficient convolution operations demonstrates unprecedented parameter-accuracy trade-offs, enabling deployment in resource-constrained edge devices without accuracy degradation.

Related papers

• Leveraging Convolutional Sparse Autoencoders for Robust Movement Classification from Low-Density sEMG [0.46976113832881716]
  This study proposes a deep learning framework for accurate gesture recognition using only two surface electromyography (sEMG) channels.<n>We present a few-shot transfer learning protocol that improved performance on unseen subjects from a baseline of 35.1% $pm$ 3.1% to 92.3% $pm$ 0.9% with minimal calibration data.
  arXiv  Detail & Related papers  (2026-01-30T14:21:46Z)
• Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models [97.55009021098554]
  This work aims to identify the key determinants of SLMs' real-device latency and offer generalizable principles and methodologies for SLM design and training.<n>We introduce a new family of hybrid SLMs, called Nemotron-Flash, which significantly advances the accuracy-efficiency frontier of state-of-the-art SLMs.
  arXiv  Detail & Related papers  (2025-11-24T08:46:36Z)
• Extreme Model Compression for Edge Vision-Language Models: Sparse Temporal Token Fusion and Adaptive Neural Compression [0.0]
  Two adaptive compression techniques are proposed to integrate algorithmic innovations with hardware-aware optimizations.<n>On event-based vision tasks, STTF reduces average token count by 84%.<n>ANC cuts FLOPs by up to 90% in low-motion scenes.
  arXiv  Detail & Related papers  (2025-11-23T15:43:00Z)
• DARN: Dynamic Adaptive Regularization Networks for Efficient and Robust Foundation Model Adaptation [0.0]
  We introduce Dynamic Adaptive Regularization Networks (DARN)<n>DARN integrates three key innovations: a lightweight Task Complexity Predictor ( TCP) that estimates per-sample difficulty, Adaptive Dropout Modulation (ADM) and Dynamic Capacity Gating (DCG)<n>In full fine-tuning (unfrozen backbone), DARN achieves a new state-of-the-art on the multi-task GeoBench benchmark (86.66% mIoU, +5.56 pp over prior SOTA).<n>In efficient adaptation (frozen backbone), DARN achieves SOTA-competitive accuracy (90.5% mIoU on Sen
  arXiv  Detail & Related papers  (2025-11-06T19:36:49Z)
• Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning [73.10669391954801]
  We present the Ring-linear model series, specifically including Ring-mini-linear-2.0 and Ring-flash-linear-2.0.<n>Both models adopt a hybrid architecture that effectively integrates linear attention and softmax attention.<n>Compared to a 32 billion parameter dense model, this series reduces inference cost to 1/10, and compared to the original Ring series, the cost is also reduced by over 50%.
  arXiv  Detail & Related papers  (2025-10-22T07:59:38Z)
• RAPID^3: Tri-Level Reinforced Acceleration Policies for Diffusion Transformer [86.57077884971478]
  Diffusion Transformers (DiTs) excel at visual generation yet remain hampered by slow sampling.<n>We introduce RAPID3: Tri-Level Reinforced Acceleration Policies for Diffusion Transformers.<n>It delivers image-wise acceleration with zero updates to the base generator.<n>It achieves nearly 3x faster sampling with competitive generation quality.
  arXiv  Detail & Related papers  (2025-09-26T13:20:52Z)
• Don't Forget the Nonlinearity: Unlocking Activation Functions in Efficient Fine-Tuning [82.16625951603315]
  NoRA replaces fixed activations with learnable rational functions and applies structured low-rank updates to numerator and denominator coefficients.<n>On vision transformers trained on CIFAR-10 and CIFAR-100, NoRA matches or exceeds full fine-tuning while updating only 0.4% of parameters.<n>NoRA constrains adaptation to a low-dimensional functional subspace, implicitly regularizing update magnitude and direction.
  arXiv  Detail & Related papers  (2025-09-16T16:47:03Z)
• Tri-Accel: Curvature-Aware Precision-Adaptive and Memory-Elastic Optimization for Efficient GPU Usage [0.6511750267058007]
  Tri-Accel is a unified optimization framework that co-adapts three acceleration strategies along with adaptive parameters during training.<n>On CIFAR-10 with ResNet-18 and EfficientNet-B0, Tri-Accel achieves up to 9.9% reduction in training time and 13.3% lower memory usage.<n>Compared to static mixed-precision training, Tri-Accel maintains 78.1% accuracy while reducing memory footprint from 0.35GB to 0.31GB on standard hardware.
  arXiv  Detail & Related papers  (2025-08-23T05:38:42Z)
• FORTRESS: Function-composition Optimized Real-Time Resilient Structural Segmentation via Kolmogorov-Arnold Enhanced Spatial Attention Networks [1.663204995903499]
  FORTRESS (Function-composition Optimized Real-Time Resilient Structural) is a new architecture that balances accuracy and speed by using a special method.<n>Fortress incorporates three key innovations: a systematic depthwise separable convolution framework, adaptive TiKAN integration, and multi-scale attention fusion.<n>The architecture achieves remarkable efficiency gains with 91% parameter reduction (31M to 2.9M), 91% computational complexity reduction (13.7 to 1.17 GFLOPs), and 3x inference speed improvement.
  arXiv  Detail & Related papers  (2025-07-16T23:17:58Z)
• Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning [126.84770886628833]
  Existing finetuning methods either tune all parameters of the pretrained model (full finetuning) or only tune the last linear layer (linear probing) We propose a new parameter-efficient finetuning method termed as SSF, representing that researchers only need to Scale and Shift the deep Features extracted by a pre-trained model to catch up with the performance full finetuning.
  arXiv  Detail & Related papers  (2022-10-17T08:14:49Z)
• FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation [81.76975488010213]
  Dense optical flow estimation plays a key role in many robotic vision tasks. Current networks often occupy large number of parameters and require heavy computation costs. Our proposed FastFlowNet works in the well-known coarse-to-fine manner with following innovations.
  arXiv  Detail & Related papers  (2021-03-08T03:09:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.