JAXenstein: Accelerated Benchmarking for First-Person Environments

• URL: http://arxiv.org/abs/2605.19926v1
• Date: Tue, 19 May 2026 14:47:55 GMT
• Title: JAXenstein: Accelerated Benchmarking for First-Person Environments
• Abstract summary: We introduce JAXenstein, an open-source benchmark that implements the Wolfenstein 3D rendering engine.<n>JAXenstein is several times faster than comparable vision-based benchmarks.
• Score: 10.578715322504737
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The progression of reinforcement learning algorithms have been driven by challenging benchmarks. The rate in which a researcher can iterate on a problem setting directly impacts the speed of algorithm development. Modern machine learning has produced tools that allow for fast and scalable algorithm development like the JAX library. With the availability of these tools, a serious bottleneck in algorithm development is the availability of large and complex domains for experimentation. Most notably, the JAX reinforcement learning ecosystem does not have any benchmarks that test visual first-person tasks; these domains are crucial for testing both exploration and an agent's ability to overcome partial observability. We introduce JAXenstein: an open-source JAX-based benchmark that implements the Wolfenstein 3D rendering engine for fast and scalable experimentation in visual first-person tasks. JAXenstein is several times faster than comparable vision-based benchmarks, and is easily extensible to more complex first-person domains.

Related papers

• TABX: A High-Throughput Sandbox Battle Simulator for Multi-Agent Reinforcement Learning [4.254850120280717]
  We introduce the Totally Accelerated Battle Simulator in JAX (TABX) for reconfigurable multi-agent tasks.<n>TABX enables massive parallelization and significantly reduces computational overhead.<n>By providing a fast, scalable, and easily customized framework, TABX serves as a scalable foundation for future research.
  arXiv  Detail & Related papers  (2026-02-02T05:34:38Z)
• A Rosetta Stone for AI Benchmarks [28.690200241767897]
  Most AI benchmarks saturate within years or even months after they are introduced, making it hard to study long-run trends in AI capabilities.<n>We build a statistical framework that stitches benchmarks together, putting model capabilities and benchmark difficulties on a single numerical scale.<n>This acts as a "Rosetta Stone", allowing us to compare models across a wide range of abilities and time, even if they are not evaluated on the same benchmarks.
  arXiv  Detail & Related papers  (2025-11-28T20:18:58Z)
• Controlling Thinking Speed in Reasoning Models [57.14541748751654]
  Human cognition operates in two modes: fast, intuitive System 1 thinking and slow, deliberate System 2 thinking.<n>In this work, we enable LRMs to approximate human intelligence through dynamic thinking speed adjustment.<n>Our approach addresses two key questions: (1) how to control thinking speed in LRMs, and (2) when to adjust it for optimal performance.
  arXiv  Detail & Related papers  (2025-07-04T16:41:06Z)
• The Automated LLM Speedrunning Benchmark: Reproducing NanoGPT Improvements [87.61432174951891]
  A critical capability toward scientific progress is the ability to reproduce existing work.<n>To evaluate the ability of AI agents to reproduce results in an active research area, we introduce the Automated LLM Speedrunning Benchmark.<n>We find that recent reasoning LLMs combined with SoTA scaffolds struggle to reimplement already-known innovations in our benchmark.
  arXiv  Detail & Related papers  (2025-06-27T17:44:32Z)
• Exposing the Copycat Problem of Imitation-based Planner: A Novel Closed-Loop Simulator, Causal Benchmark and Joint IL-RL Baseline [49.51385135697656]
  Within machine learning-based planning, imitation learning (IL) is a common algorithm.<n>It primarily learns driving policies directly from supervised trajectory data.<n>It remains challenging to determine if the learned policy truly understands fundamental driving principles.<n>This work proposes a novel closed-loop simulator supporting both imitation and reinforcement learning.
  arXiv  Detail & Related papers  (2025-04-20T18:51:26Z)
• RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation [54.707460684650584]
  Large Language Models (LLMs) demonstrate human-level capabilities in dialogue, reasoning, and knowledge retention. Current research addresses this bottleneck by equipping LLMs with external knowledge, a technique known as Retrieval Augmented Generation (RAG) RAGLAB is a modular and research-oriented open-source library that reproduces 6 existing algorithms and provides a comprehensive ecosystem for investigating RAG algorithms.
  arXiv  Detail & Related papers  (2024-08-21T07:20:48Z)
• Benchmarking Predictive Coding Networks -- Made Simple [48.652114040426625]
  We tackle the problems of efficiency and scalability for predictive coding networks (PCNs) in machine learning.<n>We propose a library, called PCX, that focuses on performance and simplicity, and use it to implement a large set of standard benchmarks.<n>We perform extensive tests on such benchmarks using both existing algorithms for PCNs, as well as adaptations of other methods popular in the bio-plausible deep learning community.
  arXiv  Detail & Related papers  (2024-07-01T10:33:44Z)
• Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning [4.067733179628694]
  Craftax is a ground-up rewrite of Crafter in JAX that runs up to 250x faster than the Python-native original. A run of PPO using 1 billion environment interactions finishes in under an hour using only a single GPU. We show that existing methods including global and episodic exploration, as well as unsupervised environment design fail to make material progress on the benchmark.
  arXiv  Detail & Related papers  (2024-02-26T18:19:07Z)
• JaxMARL: Multi-Agent RL Environments and Algorithms in JAX [105.343918678781]
  We present JaxMARL, the first open-source, Python-based library that combines GPU-enabled efficiency with support for a large number of commonly used MARL environments. Our experiments show that, in terms of wall clock time, our JAX-based training pipeline is around 14 times faster than existing approaches. We also introduce and benchmark SMAX, a JAX-based approximate reimplementation of the popular StarCraft Multi-Agent Challenge.
  arXiv  Detail & Related papers  (2023-11-16T18:58:43Z)
• ELEA -- Build your own Evolutionary Algorithm in your Browser [1.5525560291268217]
  "Experimenting and Learning toolkit for Evolutionary Algorithms" is an open source framework to experiment with evolutionary algorithms. ELEA is browser-based and allows to assemble evolutionary algorithms using drag-and-drop. ELEA is especially suitable for explorative analyses of algorithms as well as for the use in classrooms.
  arXiv  Detail & Related papers  (2023-02-13T11:32:40Z)
• EvoX: A Distributed GPU-accelerated Framework for Scalable Evolutionary Computation [40.71953374838183]
  EvoX is a computing framework tailored for automated, distributed, and heterogeneous execution of EC algorithms. At the core of EvoX lies a unique programming model to streamline the development of parallelizable EC algorithms. EvoX offers comprehensive support for a diverse set of benchmark problems, ranging from dozens of numerical test functions to hundreds of reinforcement learning tasks.
  arXiv  Detail & Related papers  (2023-01-29T15:00:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.