GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems

• URL: http://arxiv.org/abs/2609.14864v1
• Date: Mon, 14 Sep 2026 00:31:14 GMT
• Title: GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems
• Abstract summary: We predict single-sequence model throughput from GGUF metadata using roofline-shaped predictors with quantization-specific scale factors fitted on reference models.<n>The cohort comprises 318 phase-depth measurements from 53 host-file configurations on two Apple M4 Max systems and an NVIDIA GTX 5080.
• Score: 3.3498577203654816
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We predict single-sequence model throughput from GGUF metadata using roofline-shaped predictors with quantization-specific scale factors fitted on reference models. The scored cohort comprises 318 phase-depth measurements from 53 host-file configurations on two Apple M4 Max systems and an NVIDIA RTX 5080. On host-specific held-out sets of four, five, and two configurations, an active-parameter decode model obtains 13.1%, 14.4%, and 36.1% mean absolute percentage error (MAPE), versus 49.4%, 55.3%, and 51.9% when charging total parameters. Leave-one-host-out coefficients fitted on the other two systems yield 11.6%, 16.8%, and 36.0% test MAPE. A low-bit model ladder changes ordering across runtime stacks. The P2 prefill baseline gives 18.7%, 22.2%, and 108.2% test MAPE. GGUF structure helps on all three systems, but fitted efficiencies are not universal.

Related papers

• Parameter-Efficient Quantum NLP for Paraphrase Detection: Performance, Robustness, and Entanglement [0.05999777817331314]
  We evaluate a 10-qubit hybrid quantum-classical variational circuit for paraphrase detection.<n>On QQP the circuit achieves 75.53% +/- 0.75% accuracy.<n>On MRPC the optimal 2-layer variant reaches 92% of BERT-base accuracy at 54,000 lower parameter cost.<n>These results constitute the first systematic parameter-matched multi-benchmark empirical validation of hybrid variational circuits for NLP.
  arXiv  Detail & Related papers  (2026-09-13T14:06:49Z)
• Interpretable machine learning for predicting splitting strength of asphalt concrete: insights from SHAP analysis [2.5274533414341493]
  This paper presents an interpretable machine-learning framework for predicting the splitting strength (ST) of asphalt concrete.<n>Six machine-learning models, namely TabPFN, ANN, SVR, RF, XGBoost, and LightGBM, were developed and compared.<n>Results show that TabPFN delivered the best overall performance on the testing set, with the lowest RMSE of 0.28, MAE of 0.21, MAPE of 18.01%, MAD of 0.14, the highest R2 of 0.88, and the highest composite score of 0.91.
  arXiv  Detail & Related papers  (2026-08-02T03:05:54Z)
• Multi-Domain Physics-Based MDO of Multirotor UAVs: A Deterministic Framework for Discrete COTS Sizing [1.6114012813668932]
  This paper presents AeroEval, a physics-based, Multidisciplinary Design Optimization (MDO) engine.<n>Engine maps continuous sizing optima to physically purchasable, commercial off-the-shelf (COTS) components.<n>Battery mass is predicted within 7.9% on single-pack platforms.
  arXiv  Detail & Related papers  (2026-07-24T03:52:36Z)
• SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD [91.71050339479261]
  This report presents an end-to-end optimization practice on the Ascend NPU SuperPOD.<n>We develop a hierarchical optimization framework spanning model-level parallelism, computation-communication orchestration, and low-level kernel execution.<n>The resulting system achieves 34.22% Model FLOPs Utilization (MFU) with a 2.93x improvement over the open-source baseline recipe.
  arXiv  Detail & Related papers  (2026-07-22T13:49:17Z)
• MemDelta: Controlled Baselines and Hidden Confounds in Agent Memory Evaluation [8.38765074765613]
  Agent memory systems are increasingly evaluated against RAG and full-context baselines.<n> reported gains often mix changes in the memory method with changes in the language model, embedding model, or retrieval pipeline.<n>We present MemDelta, a controlled evaluation protocol that varies one component at a time on LongMemEval-S (500 questions, 50+ sessions, three model families.
  arXiv  Detail & Related papers  (2026-06-29T07:51:22Z)
• THEIA: Learning Complete Kleene Three-Valued Logic in a Pure-Neural Modular Architecture [0.0]
  THEIA is a 2.75M modular neural architecture that learns the complete Kleene three-valued logic (K3) truth table from task data without external symbolic inference or hand-encoded K3 gate primitives.<n> Transformer baselines also reach >99% on all 39 rules, and flats match THEIA on Phase-1 accuracy within 0.04pp.
  arXiv  Detail & Related papers  (2026-04-13T10:44:15Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report [180.94772271910315]
  This paper reviews the NTIRE 2024 challenge, focusing on efficient single-image super-resolution (ESR) solutions. The primary objective is to develop networks that optimize various aspects such as runtime, parameters, and FLOPs. The challenge had 262 registered participants, and 34 teams made valid submissions.
  arXiv  Detail & Related papers  (2024-04-16T07:26:20Z)
• A Meta-Learning Approach to Predicting Performance and Data Requirements [163.4412093478316]
  We propose an approach to estimate the number of samples required for a model to reach a target performance. We find that the power law, the de facto principle to estimate model performance, leads to large error when using a small dataset. We introduce a novel piecewise power law (PPL) that handles the two data differently.
  arXiv  Detail & Related papers  (2023-03-02T21:48:22Z)
• Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning [126.84770886628833]
  Existing finetuning methods either tune all parameters of the pretrained model (full finetuning) or only tune the last linear layer (linear probing) We propose a new parameter-efficient finetuning method termed as SSF, representing that researchers only need to Scale and Shift the deep Features extracted by a pre-trained model to catch up with the performance full finetuning.
  arXiv  Detail & Related papers  (2022-10-17T08:14:49Z)
• Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems [63.713297451300086]
  We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9.3B. Their subsequent distillation into smaller models ranging from 17M-170M parameters, and their application to the Natural Language Understanding (NLU) component of a virtual assistant system.
  arXiv  Detail & Related papers  (2022-06-15T20:44:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.