Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark

• URL: http://arxiv.org/abs/2103.15332v1
• Date: Mon, 29 Mar 2021 05:00:14 GMT
• Title: Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark
• Abstract summary: The NeurIPS 2020 Procgen Competition was designed as a centralized benchmark with clearly defined tasks for measuring Sample Efficiency and Generalization in Reinforcement Learning. We present the design of a centralized benchmark for Reinforcement Learning which can help measure Sample Efficiency and Generalization in Reinforcement Learning.
• Score: 22.569342580049163
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The NeurIPS 2020 Procgen Competition was designed as a centralized benchmark with clearly defined tasks for measuring Sample Efficiency and Generalization in Reinforcement Learning. Generalization remains one of the most fundamental challenges in deep reinforcement learning, and yet we do not have enough benchmarks to measure the progress of the community on Generalization in Reinforcement Learning. We present the design of a centralized benchmark for Reinforcement Learning which can help measure Sample Efficiency and Generalization in Reinforcement Learning by doing end to end evaluation of the training and rollout phases of thousands of user submitted code bases in a scalable way. We designed the benchmark on top of the already existing Procgen Benchmark by defining clear tasks and standardizing the end to end evaluation setups. The design aims to maximize the flexibility available for researchers who wish to design future iterations of such benchmarks, and yet imposes necessary practical constraints to allow for a system like this to scale. This paper presents the competition setup and the details and analysis of the top solutions identified through this setup in context of 2020 iteration of the competition at NeurIPS.

Related papers

• Measuring what matters: A scalable framework for application-level quantum benchmarking [0.21657999663857241]
  We present a scalable framework for application-level quantum benchmarking.<n>We present 13 benchmark families that reflect realistic workloads across multiple domains.<n>The benchmarks are designed to be reproducible, interpretable across stakeholder groups, and adaptable to evolving system capabilities.
  arXiv  Detail & Related papers  (2026-04-13T17:47:55Z)
• CUBE: A Standard for Unifying Agent Benchmarks [138.9831263189749]
  We propose CUBE (Common Unified Benchmark Environments), a universal protocol standard built on MCP and Gym.<n>CUBE enables any compliant platform to access any compliant benchmark for evaluation, RL training, or data generation without custom integration.
  arXiv  Detail & Related papers  (2026-03-16T18:31:37Z)
• Easy Data Unlearning Bench [53.1304932656586]
  We introduce a unified and benchmarking suite that simplifies the evaluation of unlearning algorithms.<n>By standardizing setup and metrics, it enables reproducible, scalable, and fair comparison across unlearning methods.
  arXiv  Detail & Related papers  (2026-02-18T12:20:32Z)
• Assessing and Improving the Representativeness of Code Generation Benchmarks Using Knowledge Units (KUs) of Programming Languages -- An Empirical Study [7.0773305889955616]
  Large Language Models (LLMs) have shown impressive performance in code generation.<n>LLMs must understand and apply a wide range of language concepts.<n>If the concepts exercised in benchmarks are not representative of those used in real-world projects, evaluations may yield incomplete.
  arXiv  Detail & Related papers  (2026-01-07T10:23:33Z)
• DFedReweighting: A Unified Framework for Objective-Oriented Reweighting in Decentralized Federated Learning [10.932823356075566]
  bfDFedReweighting is a unified aggregation framework designed to achieve diverse objectives in DFL systems.<n>Our results show that the appropriate combination of the target performance metric and the customized reweighting strategy ensures linear convergence.
  arXiv  Detail & Related papers  (2025-12-12T20:30:11Z)
• Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domains [97.5573252172065]
  We train a family of Automatic Reasoning Evaluators (FARE) with a simple iterative rejection-sampling supervised finetuning approach.<n>FARE-8B challenges larger specialized RL-trained evaluators and FARE-20B sets the new standard for open-source evaluators.<n>As inference-time rerankers, FARE-20B achieves near-oracle performance on MATH.
  arXiv  Detail & Related papers  (2025-10-20T17:52:06Z)
• Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models [72.52332895840279]
  GenCluster is a test-time compute framework that attains IOI gold-level performance using open-weight models.<n>We will show that GenCluster can achieve a gold medal at IOI 2025 for the first time with an open-weight model.
  arXiv  Detail & Related papers  (2025-10-16T02:19:25Z)
• Rethinking RL Evaluation: Can Benchmarks Truly Reveal Failures of RL Methods? [43.02103699297655]
  We find that training on benchmarks achieves nearly the same performance as training directly on the test sets.<n>We propose three core principles for designing more faithful benchmarks: sufficient difficulty, balanced evaluation, and distributional robustness.
  arXiv  Detail & Related papers  (2025-10-12T10:49:57Z)
• Test-time Offline Reinforcement Learning on Goal-related Experience [50.94457794664909]
  Research in foundation models has shown that performance can be substantially improved through test-time training.<n>We propose a novel self-supervised data selection criterion, which selects transitions from an offline dataset according to their relevance to the current state.<n>Our goal-conditioned test-time training (GC-TTT) algorithm applies this routine in a receding-horizon fashion during evaluation, adapting the policy to the current trajectory as it is being rolled out.
  arXiv  Detail & Related papers  (2025-07-24T21:11:39Z)
• NeurIPS 2023 LLM Efficiency Fine-tuning Competition [8.327069446234088]
  Top-performing models exhibit significant overfitting on benchmark datasets, mirroring the broader issue of benchmark overfitting on popular leaderboards. Results highlight the limitations of current benchmark-based evaluation schemes for generative models. To further research and promote, we release all competition entries, Docker files, and infrastructure.
  arXiv  Detail & Related papers  (2025-03-13T19:35:40Z)
• ProBench: Benchmarking Large Language Models in Competitive Programming [44.09445715541973]
  We propose ProBench to benchmark large language models (LLMs) in competitive programming. ProBench collects a comprehensive set of competitive programming problems from Codeforces, Luogu, and Nowcoder platforms. We assess 9 latest LLMs in competitive programming across multiple dimensions, including thought chain analysis, error type diagnosis, and reasoning depth evaluation.
  arXiv  Detail & Related papers  (2025-02-28T09:12:42Z)
• Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysis [89.60263788590893]
  Post-training Quantization (PTQ) technique has been extensively adopted for large language models (LLMs) compression. Existing algorithms focus primarily on performance, overlooking the trade-off among model size, performance, and quantization bitwidth. We provide a novel benchmark for LLMs PTQ in this paper.
  arXiv  Detail & Related papers  (2025-02-18T07:35:35Z)
• UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling [22.885385107905222]
  We introduce UniBench, a unified implementation of 50+ vision-language model (VLM) benchmarks. We showcase the utility of UniBench for measuring progress by evaluating nearly 60 publicly available vision-language models. We also release an easy-to-run UniBench code-base with the full set of 50+ benchmarks and comparisons across 59 models as well as a distilled set of benchmarks that runs in 5 minutes on a single GPU.
  arXiv  Detail & Related papers  (2024-08-09T01:41:05Z)
• Benchmarks as Microscopes: A Call for Model Metrology [76.64402390208576]
  Modern language models (LMs) pose a new challenge in capability assessment. To be confident in our metrics, we need a new discipline of model metrology.
  arXiv  Detail & Related papers  (2024-07-22T17:52:12Z)
• Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition [70.60872754129832]
  First NeurIPS competition on unlearning sought to stimulate the development of novel algorithms. Nearly 1,200 teams from across the world participated. We analyze top solutions and delve into discussions on benchmarking unlearning.
  arXiv  Detail & Related papers  (2024-06-13T12:58:00Z)
• ECBD: Evidence-Centered Benchmark Design for NLP [95.50252564938417]
  We propose Evidence-Centered Benchmark Design (ECBD), a framework which formalizes the benchmark design process into five modules. Each module requires benchmark designers to describe, justify, and support benchmark design choices. Our analysis reveals common trends in benchmark design and documentation that could threaten the validity of benchmarks' measurements.
  arXiv  Detail & Related papers  (2024-06-13T00:59:55Z)
• Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization [165.98557106089777]
  A key challenge is to enhance the capabilities of large language models (LLMs) amid a looming shortage of high-quality training data. Our study starts from an empirical strategy for the light continual training of LLMs using their original pre-training data sets. We then formalize this strategy into a principled framework of Instance-Reweighted Distributionally Robust Optimization.
  arXiv  Detail & Related papers  (2024-02-22T04:10:57Z)
• Continual Learning for Human State Monitoring [20.8311956676327]
  We propose two new CL benchmarks for Human State Monitoring. We carefully designed the benchmarks to mirror real-world environments in which new subjects are continuously added. We conducted an empirical evaluation to assess the ability of popular CL strategies to mitigate forgetting in our benchmarks.
  arXiv  Detail & Related papers  (2022-06-29T19:23:13Z)
• Open-Set Recognition: A Good Closed-Set Classifier is All You Need [146.6814176602689]
  We show that the ability of a classifier to make the 'none-of-above' decision is highly correlated with its accuracy on the closed-set classes. We use this correlation to boost the performance of the cross-entropy OSR 'baseline' by improving its closed-set accuracy. We also construct new benchmarks which better respect the task of detecting semantic novelty.
  arXiv  Detail & Related papers  (2021-10-12T17:58:59Z)
• FLEX: Unifying Evaluation for Few-Shot NLP [17.425495611344786]
  We formulate desiderata for an ideal few-shot NLP benchmark. We present FLEX, the first benchmark, public leaderboard, and framework. We also present UniFew, a simple yet strong prompt-based model for few-shot learning.
  arXiv  Detail & Related papers  (2021-07-15T07:37:06Z)
• Prioritized Level Replay [24.980249597326985]
  We introduce Prioritized Level Replay (PLR), a framework for selectively sampling the next training level. PLR significantly improves sample efficiency and generalization on Procgen Benchmark.
  arXiv  Detail & Related papers  (2020-10-08T12:46:57Z)
• Analyzing Reinforcement Learning Benchmarks with Random Weight Guessing [2.5137859989323537]
  A large number of policy networks are generated by randomly guessing their parameters, and then evaluated on the benchmark task. We show that this approach isolates the environment complexity, highlights specific types of challenges, and provides a proper foundation for the statistical analysis of the task's difficulty. We test our approach on a variety of classic control benchmarks from the OpenAI Gym, where we show that small untrained networks can provide a robust baseline for a variety of tasks.
  arXiv  Detail & Related papers  (2020-04-16T15:32:52Z)
• Target-Embedding Autoencoders for Supervised Representation Learning [111.07204912245841]
  This paper analyzes a framework for improving generalization in a purely supervised setting, where the target space is high-dimensional. We motivate and formalize the general framework of target-embedding autoencoders (TEA) for supervised prediction, learning intermediate latent representations jointly optimized to be both predictable from features as well as predictive of targets.
  arXiv  Detail & Related papers  (2020-01-23T02:37:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.