CUBE: A Standard for Unifying Agent Benchmarks

• URL: http://arxiv.org/abs/2603.15798v1
• Date: Mon, 16 Mar 2026 18:31:37 GMT
• Title: CUBE: A Standard for Unifying Agent Benchmarks
• Abstract summary: We propose CUBE (Common Unified Benchmark Environments), a universal protocol standard built on MCP and Gym.<n>CUBE enables any compliant platform to access any compliant benchmark for evaluation, RL training, or data generation without custom integration.
• Score: 138.9831263189749
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The proliferation of agent benchmarks has created critical fragmentation that threatens research productivity. Each new benchmark requires substantial custom integration, creating an "integration tax" that limits comprehensive evaluation. We propose CUBE (Common Unified Benchmark Environments), a universal protocol standard built on MCP and Gym that allows benchmarks to be wrapped once and used everywhere. By separating task, benchmark, package, and registry concerns into distinct API layers, CUBE enables any compliant platform to access any compliant benchmark for evaluation, RL training, or data generation without custom integration. We call on the community to contribute to the development of this standard before platform-specific implementations deepen fragmentation as benchmark production accelerates through 2026.

Related papers

• Flower Hub: A Reproducible Benchmarking Platform for Federated Learning in Simulation and Deployment [51.83961778298564]
  We present Flower Hub, a platform for publishing, discovering, and executing decentralized and federated applications.<n>We show how it enables reproducible benchmarking by packaging benchmarks as executable, versioned applications with standardized metadata, pinned dependencies, and explicit evaluation.
  arXiv  Detail & Related papers  (2026-08-25T20:09:44Z)
• Cluster-Aware Dual-Level Test Specification Generation for Large-Scale Automotive Software Requirements [0.769672852567215]
  "Cluster-then-Summarize" pipeline generates test specifications that satisfy Automotive SPICE SWE.6 requirements.<n>This paper presents a novel "Cluster-then-Summarize" pipeline that addresses these limitations through three-stages.
  arXiv  Detail & Related papers  (2026-06-15T18:36:57Z)
• UniDial-EvalKit: A Unified Toolkit for Evaluating Multi-Faceted Conversational Abilities [70.79422099851506]
  We present UniDial-EvalKit (UDE), a unified evaluation toolkit for assessing interactive AI systems.<n>UDE standardizes heterogeneous data formats into a universal schema, streamlines complex evaluation pipelines through a modular architecture, and aligns metric calculations under a consistent scoring interface.
  arXiv  Detail & Related papers  (2026-03-24T13:01:31Z)
• DEP: A Decentralized Large Language Model Evaluation Protocol [51.3646001384887]
  Decentralized Evaluation Protocol (DEP) is a decentralized yet unified and standardized evaluation framework.<n>By decoupling users, LLMs, and benchmarks, DEP enables modular, plug-and-play evaluation.<n>We develop DEP Toolkit, a protocol-compatible toolkit that supports features such as breakpoint resume, concurrent requests, and congestion control.
  arXiv  Detail & Related papers  (2026-03-01T16:10:16Z)
• Symfrog-512: High-Capacity Sponge-Based AEAD Cipher (1024-bit State) [0.0]
  This submission includes a complete reference implementation together with deterministic test vectors and a reproducible benchmark suite.<n>The AEAD construction is fully specified, including domain separation, rate and capacity choices, tag generation, and the exact file format used by the reference CLI.<n>Reported performance numbers are produced by the built in benchmark tool under documented hardware and compiler settings.
  arXiv  Detail & Related papers  (2026-02-19T23:39:54Z)
• Easy Data Unlearning Bench [53.1304932656586]
  We introduce a unified and benchmarking suite that simplifies the evaluation of unlearning algorithms.<n>By standardizing setup and metrics, it enables reproducible, scalable, and fair comparison across unlearning methods.
  arXiv  Detail & Related papers  (2026-02-18T12:20:32Z)
• Towards Global Retrieval Augmented Generation: A Benchmark for Corpus-Level Reasoning [50.27838512822097]
  We introduce GlobalQA, the first benchmark specifically designed to evaluate global RAG capabilities.<n>We propose GlobalRAG, a multi-tool collaborative framework that preserves structural coherence through chunk-level retrieval.<n>On the Qwen2.5-14B model, GlobalRAG achieves 6.63 F1 compared to the strongest baseline's 1.51 F1.
  arXiv  Detail & Related papers  (2025-10-30T07:29:14Z)
• Bencher: Simple and Reproducible Benchmarking for Black-Box Optimization [5.703483582960509]
  Bencher is a modular benchmarking framework for black-box optimization.<n>Each benchmark is isolated in its own virtual Python environment and accessed via a unified, version-agnostic remote procedure call (RPC) interface.<n>Bencher can be deployed locally or remotely via Docker or on high-performance computing clusters via Singularity.
  arXiv  Detail & Related papers  (2025-05-27T15:18:58Z)
• BenchAgents: Multi-Agent Systems for Structured Benchmark Creation [23.653678381444276]
  BenchAgents is a framework that automates the creation of evaluation benchmarks.<n>We use BenchAgents to create benchmarks to evaluate capabilities related to planning, constraint satisfaction, and causal reasoning.<n>We then use these benchmarks to study state-of-the-art models and extract new insights into common failure modes and model differences.
  arXiv  Detail & Related papers  (2024-10-29T22:56:18Z)
• How to Prune Your Language Model: Recovering Accuracy on the "Sparsity May Cry'' Benchmark [60.72725673114168]
  We revisit the question of accurate BERT-pruning during fine-tuning on downstream datasets. We propose a set of general guidelines for successful pruning, even on the challenging SMC benchmark.
  arXiv  Detail & Related papers  (2023-12-21T03:11:30Z)
• Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking for Everyone [45.673814384050004]
  We introduce Codabench, an open-sourced, community-driven platform for benchmarking algorithms or software agents versus datasets or tasks. A public instance of Codabench is open to everyone, free of charge, and allows benchmark organizers to compare fairly submissions.
  arXiv  Detail & Related papers  (2021-10-12T07:54:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.