EnviroLLM: Resource Tracking and Optimization for Local AI

• URL: http://arxiv.org/abs/2512.12004v1
• Date: Fri, 12 Dec 2025 19:38:10 GMT
• Title: EnviroLLM: Resource Tracking and Optimization for Local AI
• Abstract summary: This paper presents EnviroLLM, an open-source toolkit for tracking, benchmarking, and optimizing performance and energy consumption when running large language models on personal devices.<n>The system provides real-time process monitoring, benchmarking across multiple platforms, persistent storage with visualizations for longitudinal analysis, and personalized model and optimization recommendations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly deployed locally for privacy and accessibility, yet users lack tools to measure their resource usage, environmental impact, and efficiency metrics. This paper presents EnviroLLM, an open-source toolkit for tracking, benchmarking, and optimizing performance and energy consumption when running LLMs on personal devices. The system provides real-time process monitoring, benchmarking across multiple platforms (Ollama, LM Studio, vLLM, and OpenAI-compatible APIs), persistent storage with visualizations for longitudinal analysis, and personalized model and optimization recommendations. The system includes LLM-as-judge evaluations alongside energy and speed metrics, enabling users to assess quality-efficiency tradeoffs when testing models with custom prompts.

Related papers

• Profiling Lightweight Large Language Models [0.4231181719953952]
  Lightweight large language models (LLMs) are increasingly being deployed locally on personal computers.<n>This paper introduces a PTME-based experimental framework for the precision-aware profiling of lightweight LLM inference.<n>We find that static proxy descriptors approximate inference cost well but fail to predict precision.
  arXiv  Detail & Related papers  (2026-07-23T00:24:22Z)
• DABench-LLM: Standardized and In-Depth Benchmarking of Post-Moore Dataflow AI Accelerators for LLMs [18.46752801066992]
  We introduce DABench-LLM, a benchmarking framework for evaluating large language models on dataflow-based accelerators.<n>We validate DABench-LLM on three commodity dataflow accelerators, Cerebras WSE-2, SambaNova RDU, and Graphcore IPU.
  arXiv  Detail & Related papers  (2025-12-04T22:43:14Z)
• Green Metrics Tool: Measuring for fun and profit [0.0]
  Green Metrics Tool (GMT) is a novel framework for accurately measuring the resource consumption of software.<n>The tool provides a containerized, controlled, and reproducible life cycle-based approach, assessing the resource use of software during key phases.
  arXiv  Detail & Related papers  (2025-06-30T15:36:53Z)
• Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generation [96.18720164390699]
  This paper explores the use of large language models (LLMs) for annotating document utility in training retrieval and retrieval-augmented generation (RAG) systems.<n>Our results show that LLM-generated annotations enhance out-of-domain retrieval performance and improve RAG outcomes compared to models trained solely on human annotations or downstream QA metrics.
  arXiv  Detail & Related papers  (2025-04-07T16:05:52Z)
• Evaluating Personalized Tool-Augmented LLMs from the Perspectives of Personalization and Proactivity [17.723293304671877]
  We introduce a novel benchmark ETAPP for evaluating personalized tool invocation.<n>To improve the accuracy of our evaluation, we propose a key-point-based evaluation method.<n>The effectiveness of our preference-setting and key-point-based evaluation method is also validated.
  arXiv  Detail & Related papers  (2025-03-02T07:36:22Z)
• EasyJudge: an Easy-to-use Tool for Comprehensive Response Evaluation of LLMs [6.179084469089114]
  This paper presents EasyJudge, a model developed to evaluate significant language model responses. It is lightweight, precise, efficient, and user-friendly, featuring an intuitive visualization interface for ease of deployment and use.
  arXiv  Detail & Related papers  (2024-10-13T08:24:12Z)
• ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency [20.33467627548677]
  Large language models (LLMs) have surged in popularity and are extensively used in commercial applications. We conduct a detailed analysis to identify major bottlenecks that impact end-to-end latency in LLM serving systems. We then propose ScaleLLM, an optimized system for resource-efficient LLM serving.
  arXiv  Detail & Related papers  (2024-07-23T23:37:29Z)
• MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Cases [81.70591346986582]
  We introduce MobileAIBench, a benchmarking framework for evaluating Large Language Models (LLMs) and Large Multimodal Models (LMMs) on mobile devices. MobileAIBench assesses models across different sizes, quantization levels, and tasks, measuring latency and resource consumption on real devices.
  arXiv  Detail & Related papers  (2024-06-12T22:58:12Z)
• Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment [104.18002641195442]
  We introduce Self-Augmented Preference Optimization (SAPO), an effective and scalable training paradigm that does not require existing paired data. Building on the self-play concept, which autonomously generates negative responses, we further incorporate an off-policy learning pipeline to enhance data exploration and exploitation.
  arXiv  Detail & Related papers  (2024-05-31T14:21:04Z)
• MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback [78.60644407028022]
  We introduce MINT, a benchmark that evaluates large language models' ability to solve tasks with multi-turn interactions. LLMs generally benefit from tools and language feedback, with performance gains of 1-8% for each turn of tool use. LLMs evaluated, supervised instruction-finetuning (SIFT) and reinforcement learning from human feedback (RLHF) generally hurt multi-turn capabilities.
  arXiv  Detail & Related papers  (2023-09-19T15:25:42Z)
• From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning [52.257422715393574]
  We introduce a self-guided methodology for Large Language Models (LLMs) to autonomously discern and select cherry samples from open-source datasets. Our key innovation, the Instruction-Following Difficulty (IFD) metric, emerges as a pivotal metric to identify discrepancies between a model's expected responses and its intrinsic generation capability.
  arXiv  Detail & Related papers  (2023-08-23T09:45:29Z)
• Cheaply Evaluating Inference Efficiency Metrics for Autoregressive Transformer APIs [66.30706841821123]
  Large language models (LLMs) power many state-of-the-art systems in natural language processing. LLMs are extremely computationally expensive, even at inference time. We propose a new metric for comparing inference efficiency across models.
  arXiv  Detail & Related papers  (2023-05-03T21:51:42Z)
• Automated Machine Learning: A Case Study on Non-Intrusive Appliance Load Monitoring [81.06807079998117]
  We propose a novel approach to enable Automated Machine Learning (AutoML) for Non-Intrusive Appliance Load Monitoring (NIALM)<n>NIALM offers a cost-effective alternative to smart meters for measuring the energy consumption of electric devices and appliances.
  arXiv  Detail & Related papers  (2022-03-06T10:12:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.