Energy Efficient Software Hardware CoDesign for Machine Learning: From TinyML to Large Language Models

• URL: http://arxiv.org/abs/2603.23668v1
• Date: Tue, 24 Mar 2026 19:10:42 GMT
• Title: Energy Efficient Software Hardware CoDesign for Machine Learning: From TinyML to Large Language Models
• Abstract summary: This work reviews energy efficient software hardware codesign methods spanning edge inference and training to datacenter-scale LLM serving.<n>We distill common design levers and trade-offs, and highlight recurring gaps including limited cross-platform generalization, large and costly co-design search spaces, and inconsistent benchmarking across workloads and deployment settings.<n>Finally, we outline a hierarchical decomposition perspective that maps optimization strategies to computational roles and supports incremental adaptation, offering practical guidance for building energy and carbon aware ML systems.
• Score: 0.4852108302782534
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid deployment of machine learning across platforms from milliwatt-class TinyML devices to large language models has made energy efficiency a primary constraint for sustainable AI. Across these scales, performance and energy are increasingly limited by data movement and memory-system behavior rather than by arithmetic throughput alone. This work reviews energy efficient software hardware codesign methods spanning edge inference and training to datacenter-scale LLM serving, covering accelerator architectures (e.g., ASIC/FPGA dataflows, processing-/compute-in-memory designs) and system-level techniques (e.g., partitioning, quantization, scheduling, and runtime adaptation). We distill common design levers and trade-offs, and highlight recurring gaps including limited cross-platform generalization, large and costly co-design search spaces, and inconsistent benchmarking across workloads and deployment settings. Finally, we outline a hierarchical decomposition perspective that maps optimization strategies to computational roles and supports incremental adaptation, offering practical guidance for building energy and carbon aware ML systems.

Related papers

• LLM Inference in a Flash! [66.43896490006965]
  Large Language Models (LLMs) have shown impressive capabilities across a range of natural language processing tasks.<n>We present an end-to-end integer-only quantization approach to eliminate expensive floating-point computations.
  arXiv  Detail & Related papers  (2026-09-14T18:04:30Z)
• A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Design [71.32556809242666]
  The rapid expansion of large-scale AI models has led to significant performance breakthroughs across diverse domains.<n>It has also raised critical concerns regarding computational costs, energy consumption, and environmental sustainability.<n>This survey aims to offer a holistic roadmap toward sustainable, scalable, and socially responsible development of large models.
  arXiv  Detail & Related papers  (2026-07-10T04:02:22Z)
• OpenCompass: A Universal Evaluation Platform for Large Language Models [62.59671563145442]
  General-purpose large language models (LLMs) have become a critical link in advancing technological development.<n>Mainstream benchmark datasets face challenges such as the diversity of task types, inconsistent evaluation criteria, and fragmentation of data and processing.<n>This paper proposes and open-source Open LLMs, a one-stop, scalable, and high-concurrency general-purpose evaluation platform.
  arXiv  Detail & Related papers  (2026-05-19T02:50:11Z)
• MAHL: Multi-Agent LLM-Guided Hierarchical Chiplet Design with Adaptive Debugging [30.305211001929496]
  Large Language Models (LLMs) are promising to extend their abilities to 2.5D integration.<n>LLMs face challenges such as flatten design, high validation cost and imprecise parameter optimization.<n>We propose MAHL, a hierarchical LLM-based chiplet design generation framework.
  arXiv  Detail & Related papers  (2025-08-08T05:47:31Z)
• Scaling Intelligence: Designing Data Centers for Next-Gen Language Models [0.6168147650666682]
  Large Language Models (LLMs), such as GPT-4 with 1.8 trillion parameters, demand a fundamental rethinking of data center architecture.<n>Our work provides a comprehensive co-design framework that jointly explores FLOPS, bandwidth and capacity, multiple network topologies.<n>We quantify the benefits of overlapping compute and communication, leveraging hardware-accelerated collectives, widening the scale-out domain, and increasing memory capacity.
  arXiv  Detail & Related papers  (2025-06-17T22:29:37Z)
• Energy Considerations of Large Language Model Inference and Efficiency Optimizations [28.55549828393871]
  As large language models (LLMs) scale in size and adoption, their computational and environmental costs continue to rise.<n>We systematically analyze the energy implications of common inference efficiency optimizations across diverse NLP and AI workloads.<n>Our findings reveal that the proper application of relevant inference efficiency optimizations can reduce total energy use by up to 73% from unoptimized baselines.
  arXiv  Detail & Related papers  (2025-04-24T15:45:05Z)
• PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing [48.30406812516552]
  We introduce the PLM, a Peripheral Language Model, developed through a co-design process that jointly optimize model architecture and edge system constraints.<n>PLM employs a Multi-head Latent Attention mechanism and employs the squared ReLU activation function to encourage sparsity, thereby reducing peak memory footprint.<n> evaluation results demonstrate that PLM outperforms existing small language models trained on publicly available data.
  arXiv  Detail & Related papers  (2025-03-15T15:11:17Z)
• Performance and Power: Systematic Evaluation of AI Workloads on Accelerators with CARAML [0.0]
  The CARAML benchmark suite is employed to assess performance and energy consumption during the training of large language models and computer vision models. CarAML provides a compact, automated, and reproducible framework for assessing the performance and energy of ML workloads.
  arXiv  Detail & Related papers  (2024-09-19T12:43:18Z)
• Machine Learning Insides OptVerse AI Solver: Design Principles and Applications [74.67495900436728]
  We present a comprehensive study on the integration of machine learning (ML) techniques into Huawei Cloud's OptVerse AI solver. We showcase our methods for generating complex SAT and MILP instances utilizing generative models that mirror multifaceted structures of real-world problem. We detail the incorporation of state-of-the-art parameter tuning algorithms which markedly elevate solver performance.
  arXiv  Detail & Related papers  (2024-01-11T15:02:15Z)
• Energy-efficient Task Adaptation for NLP Edge Inference Leveraging Heterogeneous Memory Architectures [68.91874045918112]
  adapter-ALBERT is an efficient model optimization for maximal data reuse across different tasks. We demonstrate the advantage of mapping the model to a heterogeneous on-chip memory architecture by performing simulations on a validated NLP edge accelerator.
  arXiv  Detail & Related papers  (2023-03-25T14:40:59Z)
• COMET: A Comprehensive Cluster Design Methodology for Distributed Deep Learning Training [42.514897110537596]
  Modern Deep Learning (DL) models have grown to sizes requiring massive clusters of specialized, high-end nodes to train. designing such clusters to maximize both performance and utilization--to amortize their steep cost--is a challenging task. We introduce COMET, a holistic cluster design methodology and workflow to jointly study the impact of parallelization strategies and key cluster resource provisioning on the performance of distributed DL training.
  arXiv  Detail & Related papers  (2022-11-30T00:32:37Z)
• Scalable Deep-Learning-Accelerated Topology Optimization for Additively Manufactured Materials [4.221095652322005]
  Topology optimization (TO) is a popular and powerful computational approach for designing novel structures, materials, and devices. To address these issues, we propose a general scalable deep-learning (DL) based TO framework, referred to as SDL-TO. Our framework accelerates TO by learning the iterative history data and simultaneously training on the mapping between the given design and its gradient.
  arXiv  Detail & Related papers  (2020-11-28T17:38:31Z)
• Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights [18.04657939198617]
  This paper provides a comprehensive survey on the efficient execution of sparse and irregular tensor computations of machine learning models on hardware accelerators. It analyzes different hardware designs and acceleration techniques and analyzes them in terms of hardware and execution costs. The takeaways from this paper include: understanding the key challenges in accelerating sparse, irregular-shaped, and quantized tensors.
  arXiv  Detail & Related papers  (2020-07-02T04:08:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.