Automated Evolutionary Optimization for Resource-Efficient Neural Network Training

• URL: http://arxiv.org/abs/2510.09566v1
• Date: Fri, 10 Oct 2025 17:17:49 GMT
• Title: Automated Evolutionary Optimization for Resource-Efficient Neural Network Training
• Abstract summary: We have developed a new automated machine learning (AutoML) framework, Efficient Training with Robust Automation (PETRA)<n>It applies evolutionary optimization to model architecture and training strategy.<n>PETRA includes pruning, quantization, and loss regularization.
• Score: 0.3905614402386577
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There are many critical challenges in optimizing neural network models, including distributed computing, compression techniques, and efficient training, regardless of their application to specific tasks. Solving such problems is crucial because the need for scalable and resource-efficient models is increasing. To address these challenges, we have developed a new automated machine learning (AutoML) framework, Parameter Efficient Training with Robust Automation (PETRA). It applies evolutionary optimization to model architecture and training strategy. PETRA includes pruning, quantization, and loss regularization. Experimental studies on real-world data with financial event sequences, as well as image and time-series -- benchmarks, demonstrate PETRA's ability to improve neural model performance and scalability -- namely, a significant decrease in model size (up to 75%) and latency (up to 33%), and an increase in throughput (by 13%) without noticeable degradation in the target metric.

Related papers

• Learnware for CSI Feedback: Scene-specific Small Models Can Do Big [118.52196588174786]
  Intelligent channel state information (CSI) feedback is essential for realizing the high capacity and spectral efficiency goals of future 6G systems.<n>Existing deep learning solutions face a trade-off between model generalization and scenario-specific performance.<n>We introduce a model repository-based deployment framework in which a central AI data center maintains a catalog of scene-specific CSI models.
  arXiv  Detail & Related papers  (2026-08-18T13:26:34Z)
• Beyond Backbone Backpropagation: A Decoupled Strategy for Efficient Transfer Learning [1.8798171797988195]
  Deep learning models achieve state-of-the-art image classification but face deployment challenges due to computational costs and energy demands.<n>We propose a lightweight training strategy that adapts normalization layers of the model to the new domain.<n>Our approach significantly reduces the required training time with only a marginal accuracy trade-off, often matching or surpassing baseline performance.
  arXiv  Detail & Related papers  (2026-06-13T13:46:24Z)
• Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategies [66.83950068218033]
  Scaling Laws demonstrate that scaling model parameters and training data enhances learning performance.<n>Despite its potential to improve performance, the integration of scaling laws into deep reinforcement learning has not been fully realized.<n>This review addresses this gap by systematically analyzing scaling strategies in three dimensions: data, network, and training budget.
  arXiv  Detail & Related papers  (2025-08-05T08:03:12Z)
• Variance-Based Pruning for Accelerating and Compressing Trained Networks [46.498278084317704]
  Variance-Based Pruning is a simple and structured one-shot pruning technique for efficiently compressing networks.<n>On ImageNet-1k recognition tasks, we demonstrate that directly after pruning DeiT-Base retains over 70% of its original performance.
  arXiv  Detail & Related papers  (2025-07-17T10:54:17Z)
• Auto-Compressing Networks [51.221103189527014]
  We introduce Auto-compression Networks (ACNs), an architectural variant where long feedforward connections from each layer replace traditional short residual connections.<n>We show that ACNs exhibit enhanced noise compared to residual networks, superior performance in low-data settings, and mitigate catastrophic forgetting.<n>These findings establish ACNs as a practical approach to developing efficient neural architectures.
  arXiv  Detail & Related papers  (2025-06-11T13:26:09Z)
• A Multi-Fidelity Graph U-Net Model for Accelerated Physics Simulations [1.2430809884830318]
  We propose a novel GNN architecture, Multi-Fidelity U-Net, that utilizes the advantages of the multi-fidelity methods for enhancing the performance of the GNN model.<n>We show that the proposed approach performs significantly better in accuracy and data requirement.<n>We also present Multi-Fidelity U-Net Lite, a faster version of the proposed architecture, with 35% faster training, with 2 to 5% reduction in accuracy.
  arXiv  Detail & Related papers  (2024-12-19T20:09:38Z)
• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch [72.26822499434446]
  Auto-Train-Once (ATO) is an innovative network pruning algorithm designed to automatically reduce the computational and storage costs of DNNs. We provide a comprehensive convergence analysis as well as extensive experiments, and the results show that our approach achieves state-of-the-art performance across various model architectures.
  arXiv  Detail & Related papers  (2024-03-21T02:33:37Z)
• Diffusion-Based Neural Network Weights Generation [80.89706112736353]
  D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation. Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
  arXiv  Detail & Related papers  (2024-02-28T08:34:23Z)
• EsaCL: Efficient Continual Learning of Sparse Models [10.227171407348326]
  Key challenge in the continual learning setting is to efficiently learn a sequence of tasks without forgetting how to perform previously learned tasks. We propose a new method for efficient continual learning of sparse models (EsaCL) that can automatically prune redundant parameters without adversely impacting the model's predictive power.
  arXiv  Detail & Related papers  (2024-01-11T04:59:44Z)
• Split-Boost Neural Networks [1.1549572298362787]
  We propose an innovative training strategy for feed-forward architectures - called split-boost. Such a novel approach ultimately allows us to avoid explicitly modeling the regularization term. The proposed strategy is tested on a real-world (anonymized) dataset within a benchmark medical insurance design problem.
  arXiv  Detail & Related papers  (2023-09-06T17:08:57Z)
• RLFlow: Optimising Neural Network Subgraph Transformation with World Models [0.0]
  We propose a model-based agent which learns to optimise the architecture of neural networks by performing a sequence of subgraph transformations to reduce model runtime. We show our approach can match the performance of state of the art on common convolutional networks and outperform those by up to 5% on transformer-style architectures.
  arXiv  Detail & Related papers  (2022-05-03T11:52:54Z)
• A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuning [90.44219200633286]
  We propose a simple yet very effective adversarial fine-tuning approach based on a $textitslow start, fast decay$ learning rate scheduling strategy. Experimental results show that the proposed adversarial fine-tuning approach outperforms the state-of-the-art methods on CIFAR-10, CIFAR-100 and ImageNet datasets.
  arXiv  Detail & Related papers  (2020-12-25T20:50:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.