GLM-5 Serving Parameter Tuning for OpenClaw: Single-Deployment MaaS Inference Optimization for Long-Context Agent Workloads

• URL: http://arxiv.org/abs/2607.02518v1
• Date: Mon, 27 Apr 2026 11:12:33 GMT
• Title: GLM-5 Serving Parameter Tuning for OpenClaw: Single-Deployment MaaS Inference Optimization for Long-Context Agent Workloads
• Abstract summary: OpenClaw requests are dominated by long, tool-augmented prefixes, including system prompts, conversation history, and tool outputs fed back into the context window.<n>For this workload, with about 28k-30k input tokens and 500 output tokens per request, serving quality is governed by throughput, TTFT, and tail latency rather than short-prompt throughput alone.
• Score: 6.400220635933459
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: OpenClaw requests are dominated by long, tool-augmented prefixes, including system prompts, conversation history, and tool outputs fed back into the context window. For this workload, with about 28k-30k input tokens and 500 output tokens per request, serving quality is governed by throughput, TTFT, and tail latency rather than short-prompt throughput alone. This report studies GLM-5 serving-parameter tuning within a MaaS multi-model inference optimization architecture. The scope is the Single-Node Optimization block of the inference-optimization layer, where chunked prefill, tensor parallelism (TP), pipeline parallelism (PP), and request concurrency are tuned for one GLM-5 serving deployment; in this report, "Single-Node Optimization" denotes the architecture block, while experiments run on a two-node, sixteen-GPU cluster. Within the tested space, the best configuration is chunked-prefill-size=3072, tp=4, pp-size=4, and max-running-requests=24. Compared with the conservative 2048/4/4/16 baseline, it increases request throughput from 0.43 to 0.48 req/s and total token throughput from 9029.64 to 9993.23 tok/s, while reducing average TTFT from 8.98 to 6.69 s and latency P90 from 40.23 to 32.64 s. Under the same hardware footprint, this corresponds to an estimated 10.4% lower serving cost per request and 9.6% lower cost per token. The results show that the optimum is workload-specific: larger chunk sizes and deeper queueing do not monotonically improve performance. We therefore recommend 3072 / tp4 / pp4 / max24 as the default OpenClaw deployment profile.

Related papers

• Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models [65.49950267695267]
  We introduce Omni2LoRA, a framework for efficient parametric memory compression via coherence-preserving context distillation.<n>By converting multimodal memory into a fixed-budget, reusable parameter state, our method drives answer-time multimodal-token load to zero.
  arXiv  Detail & Related papers  (2026-08-10T07:49:10Z)
• Lower Bounds for PIR with Preprocessing from Blackbox Cryptography [57.20693707878285]
  We study the limits of single-server private information retrieval (PIR) with preprocessing.<n>Our work presents computation lower bounds for any single-server PIR with preprocessing.
  arXiv  Detail & Related papers  (2026-07-07T16:12:48Z)
• Think Before You Grid-Search: Floor-First Triage for LLM Serving [0.0]
  LLM serving optimization typically benchmarks many configurations and reaches for heavy profilers when latency targets are missed.<n>We argue for the reverse discipline: estimation is the analytical layer of profiling.
  arXiv  Detail & Related papers  (2026-07-07T06:11:54Z)
• ChunkFT: Byte-Streamed Optimization for Memory-Efficient Full Fine-Tuning [58.54940026861599]
  textscChunkFT is a memory-efficient fine-tuning framework.<n>textscChunkFT enables gradient computation for arbitrary sub-tensors without modifying the network architecture.<n>textscChunkFT consistently outperforms existing memory-efficient baselines.
  arXiv  Detail & Related papers  (2026-05-20T13:44:44Z)
• Dual-Pool Token-Budget Routing for Cost-Efficient and Reliable LLM Serving [9.457255218406333]
  Production vLLM fleets typically provision each instance for the worst-case context length.<n>In practice, 80-95% of requests are short, yet are served under configurations optimized for long contexts.<n>We propose dual-pool token-budget routing, a lightweight dispatch mechanism that partitions a homogeneous fleet into two specialized pools.
  arXiv  Detail & Related papers  (2026-04-09T10:47:20Z)
• Architecture-Aware LLM Inference Optimization on AMD Instinct GPUs: A Comprehensive Benchmark and Deployment Study [0.0]
  Cross-architecture evaluation of production LLM inference on AMD Instinct MI325X GPUs.<n> benchmarking four models spanning 235B to 1 trillion parameters across three architectural families.
  arXiv  Detail & Related papers  (2026-02-27T13:21:48Z)
• StreamServe: Adaptive Speculative Flows for Low-Latency Disaggregated LLM Serving [2.1072136327189672]
  StreamServe is a disaggregated prefill decode serving architecture.<n>StreamServe reduces latency by 11 to 18 times relative to tensor parallel vLLM baselines.<n>It reaches throughput up to 2235 tokens per second on summarization tasks.
  arXiv  Detail & Related papers  (2026-02-11T21:03:47Z)
• EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference [49.94169109038806]
  This paper introduces EPS-MoE, a novel expert pipeline scheduler for MoE that surpasses the existing parallelism schemes.<n>Our results demonstrate at most 52.4% improvement in prefill throughput compared to existing parallel inference methods.
  arXiv  Detail & Related papers  (2024-10-16T05:17:49Z)
• HiRE: High Recall Approximate Top-$k$ Estimation for Efficient LLM Inference [68.59839755875252]
  HiRE comprises of two novel components: (i) a compression scheme to cheaply predict top-$k$ rows/columns with high recall, followed by full computation restricted to the predicted subset, and (ii) DA-TOP-$k$: an efficient multi-device approximate top-$k$ operator. We demonstrate that on a one billion parameter model, HiRE applied to both the softmax as well as feedforward layers, achieves almost matching pretraining and downstream accuracy, and speeds up inference latency by $1.47times$ on a single TPUv5e device.
  arXiv  Detail & Related papers  (2024-02-14T18:04:36Z)
• Non-Parametric Adaptive Network Pruning [125.4414216272874]
  We introduce non-parametric modeling to simplify the algorithm design. Inspired by the face recognition community, we use a message passing algorithm to obtain an adaptive number of exemplars. EPruner breaks the dependency on the training data in determining the "important" filters.
  arXiv  Detail & Related papers  (2021-01-20T06:18:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.