One Search Fits All: Pareto-Optimal Eco-Friendly Model Selection

• URL: http://arxiv.org/abs/2505.01468v1
• Date: Fri, 02 May 2025 05:59:21 GMT
• Title: One Search Fits All: Pareto-Optimal Eco-Friendly Model Selection
• Authors: Filippo Betello, Antonio Purificato, Vittoria Vineis, Gabriele Tolomei, Fabrizio Silvestri,
• Abstract summary: We introduce GREEN (Guided Recommendations of Energy-Efficient Networks), a novel, inference-time approach for recommending model configurations.<n>Our approach directly addresses the limitations of current eco-efficient neural architecture search methods.<n>We show that our method successfully identifies energy-efficient configurations while ensuring competitive performance.
• Score: 5.8822780483882315
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The environmental impact of Artificial Intelligence (AI) is emerging as a significant global concern, particularly regarding model training. In this paper, we introduce GREEN (Guided Recommendations of Energy-Efficient Networks), a novel, inference-time approach for recommending Pareto-optimal AI model configurations that optimize validation performance and energy consumption across diverse AI domains and tasks. Our approach directly addresses the limitations of current eco-efficient neural architecture search methods, which are often restricted to specific architectures or tasks. Central to this work is EcoTaskSet, a dataset comprising training dynamics from over 1767 experiments across computer vision, natural language processing, and recommendation systems using both widely used and cutting-edge architectures. Leveraging this dataset and a prediction model, our approach demonstrates effectiveness in selecting the best model configuration based on user preferences. Experimental results show that our method successfully identifies energy-efficient configurations while ensuring competitive performance.

Related papers

• A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning [5.280048850098648]
  Foundation models have revolutionized artificial intelligence by providing robust, versatile architectures pre-trained on large-scale datasets.<n>Adapting these massive models to specific downstream tasks requires fine-tuning, which can be prohibitively expensive in computational resources.<n>This survey provides a comprehensive review of the integration of PEFT techniques within federated learning environments.
  arXiv  Detail & Related papers  (2025-04-29T18:18:39Z)
• Energy Considerations of Large Language Model Inference and Efficiency Optimizations [28.55549828393871]
  As large language models (LLMs) scale in size and adoption, their computational and environmental costs continue to rise.<n>We systematically analyze the energy implications of common inference efficiency optimizations across diverse NLP and AI workloads.<n>Our findings reveal that the proper application of relevant inference efficiency optimizations can reduce total energy use by up to 73% from unoptimized baselines.
  arXiv  Detail & Related papers  (2025-04-24T15:45:05Z)
• Fine-tune Smarter, Not Harder: Parameter-Efficient Fine-Tuning for Geospatial Foundation Models [16.522696273752835]
  Earth observation is crucial for monitoring environmental changes, responding to disasters, and managing natural resources.<n>Foundation models facilitate remote sensing image analysis to retrieve relevant geoinformation accurately and efficiently.<n>As these models grow in size, fine-tuning becomes increasingly challenging due to associated computational resources and costs.
  arXiv  Detail & Related papers  (2025-04-24T09:37:02Z)
• A Survey on Inference Optimization Techniques for Mixture of Experts Models [50.40325411764262]
  Large-scale Mixture of Experts (MoE) models offer enhanced model capacity and computational efficiency through conditional computation.<n> deploying and running inference on these models presents significant challenges in computational resources, latency, and energy efficiency.<n>This survey analyzes optimization techniques for MoE models across the entire system stack.
  arXiv  Detail & Related papers  (2024-12-18T14:11:15Z)
• Enhanced Bayesian Optimization via Preferential Modeling of Abstract Properties [49.351577714596544]
  We propose a human-AI collaborative Bayesian framework to incorporate expert preferences about unmeasured abstract properties into surrogate modeling. We provide an efficient strategy that can also handle any incorrect/misleading expert bias in preferential judgments.
  arXiv  Detail & Related papers  (2024-02-27T09:23:13Z)
• A Bayesian Approach to Robust Inverse Reinforcement Learning [54.24816623644148]
  We consider a Bayesian approach to offline model-based inverse reinforcement learning (IRL) The proposed framework differs from existing offline model-based IRL approaches by performing simultaneous estimation of the expert's reward function and subjective model of environment dynamics. Our analysis reveals a novel insight that the estimated policy exhibits robust performance when the expert is believed to have a highly accurate model of the environment.
  arXiv  Detail & Related papers  (2023-09-15T17:37:09Z)
• A Comparative Study of Machine Learning Algorithms for Anomaly Detection in Industrial Environments: Performance and Environmental Impact [62.997667081978825]
  This study seeks to address the demands of high-performance machine learning models with environmental sustainability. Traditional machine learning algorithms, such as Decision Trees and Random Forests, demonstrate robust efficiency and performance. However, superior outcomes were obtained with optimised configurations, albeit with a commensurate increase in resource consumption.
  arXiv  Detail & Related papers  (2023-07-01T15:18:00Z)
• Learning to Continuously Optimize Wireless Resource in a Dynamic Environment: A Bilevel Optimization Perspective [52.497514255040514]
  This work develops a new approach that enables data-driven methods to continuously learn and optimize resource allocation strategies in a dynamic environment. We propose to build the notion of continual learning into wireless system design, so that the learning model can incrementally adapt to the new episodes. Our design is based on a novel bilevel optimization formulation which ensures certain fairness" across different data samples.
  arXiv  Detail & Related papers  (2021-05-03T07:23:39Z)
• Learning to Continuously Optimize Wireless Resource In Episodically Dynamic Environment [55.91291559442884]
  This work develops a methodology that enables data-driven methods to continuously learn and optimize in a dynamic environment. We propose to build the notion of continual learning into the modeling process of learning wireless systems. Our design is based on a novel min-max formulation which ensures certain fairness" across different data samples.
  arXiv  Detail & Related papers  (2020-11-16T08:24:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.