WorldBench: Culturally Grounded Benchmark for Multilingual Agents

• URL: http://arxiv.org/abs/2609.01056v1
• Date: Tue, 01 Sep 2026 10:53:07 GMT
• Title: WorldBench: Culturally Grounded Benchmark for Multilingual Agents
• Abstract summary: Existing benchmarks rarely test state preservation, performance across languages, and application to realistic, grounded scenarios.<n>We present WorldBench: a comprehensive, multilingual benchmark, where agents can act in a sandbox via structured actions.<n>WorldBench comprises 1,600 tasks across seven languages and eight cultures, filtered and refined through feedback from human annotators.
• Score: 69.76002914143531
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the growing use of LLM-powered agents to solve multi-step tasks in complex environments, existing benchmarks rarely test state preservation, performance across languages, and application to realistic, grounded scenarios. To address these concerns, we present WorldBench: a comprehensive, multilingual benchmark of genuine, persona-grounded everyday workflows, where agents can act in a sandbox via structured actions. WorldBench comprises 1,600 tasks across seven languages and eight cultures, filtered and refined through feedback from human annotators with language- and culture-specific expertise. For evaluation, we extend metrics from previous works and introduce Constrained Task Success (CTS), which combines natural language instructions and testbeds to score task completion, minimal modification, and other complementary metrics through deterministic and LLM-as-a-Judge evaluations. Our experiments show that frontier models reach only 49.2% CTS, with all models demonstrating large gaps between correctness and environment preservation. We thereby show that current agents remain brittle in multilingual, agentic scenarios, especially for long-horizon tasks and under state-preservation constraints

Related papers

• PolyWorkBench: Benchmarking Multilingual Long-Horizon LLM Agents [72.96251271825038]
  Large language model (LLM) agents have shown strong performance in long-horizon tasks that require planning, tool use, and interaction with external environments.<n>Most existing benchmarks implicitly assume a monolingual setting, where the entire execution process is conducted within a single language.<n>We introduce PolyWorkBench, a benchmark for evaluating agents on multilingual long-horizon workplace work.
  arXiv  Detail & Related papers  (2026-07-07T08:50:09Z)
• CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks [64.22418143822016]
  CulturALL is a benchmark to assess large language models' multilingual and multicultural competence on grounded tasks.<n>It contains 2,610 samples in 14 languages from 51 regions, distributed across 16 topics to capture the full breadth of grounded tasks.<n>Experiments show that the best LLM achieves 44.48% accuracy on CulturALL, underscoring substantial room for improvement.
  arXiv  Detail & Related papers  (2026-04-21T09:21:46Z)
• Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces [126.23612941699565]
  Terminal-Bench 2.0 is a benchmark composed of 89 tasks in computer terminal environments inspired by problems from real world.<n>We show that frontier models and agents score less than 65% on the benchmark.<n>We publish the dataset and evaluation harness to assist developers and researchers in future work at https://www.tbench.ai/.
  arXiv  Detail & Related papers  (2026-01-17T01:29:30Z)
• Bridging the Knowledge Void: Inference-time Acquisition of Unfamiliar Programming Languages for Coding Tasks [22.908904483320953]
  Large Language Models (LLMs) in coding tasks are often a reflection of their extensive pre-training corpora.<n>We propose ILA-agent, a general ILA framework that equips LLMs with a set of behavioral primitives.<n>We instantiate ILA-agent for Cangjie and evaluate its performance across code generation, translation, and program repair tasks.
  arXiv  Detail & Related papers  (2026-01-16T09:06:47Z)
• Ticket-Bench: A Kickoff for Multilingual and Regionalized Agent Evaluation [4.563830993050022]
  We introduce Ticket-Bench, a benchmark for multilingual agent evaluation in task-oriented scenarios.<n>Ticket-Bench simulates the domain of soccer ticket purchases across six major languages: Portuguese, English, Spanish, German, Italian, and French.<n>We evaluate a wide range of commercial and open-source LLMs, measuring function-calling accuracy and consistency across languages.
  arXiv  Detail & Related papers  (2025-09-17T23:13:47Z)
• IberBench: LLM Evaluation on Iberian Languages [2.3034630097498883]
  Large Language Models (LLMs) are difficult to evaluate comprehensively, particularly for languages other than English.<n>We present IberBench, a benchmark designed to assess LLM performance on both fundamental and industry-relevant NLP tasks.<n>We evaluate 23 LLMs ranging from 100 million to 14 billion parameters and provide empirical insights into their strengths and limitations.
  arXiv  Detail & Related papers  (2025-04-23T17:48:25Z)
• EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents [63.43699771428243]
  EmbodiedBench is an extensive benchmark designed to evaluate vision-driven embodied agents.<n>We evaluated 24 leading proprietary and open-source MLLMs within EmbodiedBench.<n> MLLMs excel at high-level tasks but struggle with low-level manipulation, with the best model, GPT-4o, scoring only 28.9% on average.
  arXiv  Detail & Related papers  (2025-02-13T18:11:34Z)
• P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs [84.24644520272835]
  We introduce P-MMEval, a large-scale benchmark covering effective fundamental and capability-specialized datasets.<n>P-MMEval delivers consistent language coverage across various datasets and provides parallel samples.<n>We conduct extensive experiments on representative multilingual model series to compare performances across models and tasks.
  arXiv  Detail & Related papers  (2024-11-14T01:29:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.