Tape: A Cellular Automata Benchmark for Evaluating Rule-Shift Generalization in Reinforcement Learning

• URL: http://arxiv.org/abs/2601.04695v1
• Date: Thu, 08 Jan 2026 08:05:42 GMT
• Title: Tape: A Cellular Automata Benchmark for Evaluating Rule-Shift Generalization in Reinforcement Learning
• Abstract summary: We present Tape, a controlled reinforcement-learning benchmark designed to isolate out-of-distribution (OOD) failure under latent rule shifts.<n>We provide (i) standardized OOD protocols, (ii) statistical reporting requirements, and (iii) information-theoretic identities connecting entropy reduction to conditional mutual information.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Tape, a controlled reinforcement-learning benchmark designed to isolate out-of-distribution (OOD) failure under latent rule shifts.Tape is derived from one-dimensional cellular automata, enabling precise train/test splits where observation and action spaces are held fixed while transition rules change. Using a reproducible evaluation pipeline, we compare model-free baselines, model-based planning with learned world models, and task-inference (meta-RL) methods. A consistent pattern emerges: methods that are strong in-distribution (ID) can collapse under heldout-rule OOD, and high-variance OOD evaluation can make rankings unstable unless experiments are sufficiently replicated.We provide (i) standardized OOD protocols, (ii) statistical reporting requirements (seeds, confidence intervals, and hypothesis tests), and (iii) information-theoretic identities connecting entropy reduction to conditional mutual information and expected posterior KL divergence, clarifying what "uncertainty reduction" objectives can and cannot guarantee under rule shifts.

Related papers

• Purified OPSD: On-Policy Self-Distillation Without Losing How to Think [52.879387744920415]
  We find that on-policy self-distillation (OPSD) consistently fails on long chain-of-thought (long-CoT) reasoning models.<n>We propose a two-step solution: first, we construct a reference-only teacher to isolate the non-transferable component of the supervision signal.<n>Second, we use pointwise mutual information (PMI) as the mechanism to transform this residual into a well-formed PMI target distribution.
  arXiv  Detail & Related papers  (2026-07-02T14:33:07Z)
• A Formula-Driven Survey and Research Agenda for On-Policy Distillation [4.397842507533513]
  This survey studies OPD as a feedback-to-update problem rather than a single loss family.<n>We develop a formula-driven taxonomy from two routes -- direct distributional losses and policy-gradient-style log-ratio updates.
  arXiv  Detail & Related papers  (2026-06-22T03:09:21Z)
• Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration [72.0672328514289]
  Multimodal learning often grapples with the challenge of low-quality data.<n>We propose a unified framework, termed Conformal Predictive Self-Calibration.<n>Our framework consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-05T14:48:52Z)
• Geometrically Constrained Outlier Synthesis [4.254099382808598]
  Deep neural networks for image classification often exhibit overconfidence on out-of-distribution (OOD) samples.<n>We introduce Geometrically Constrained Outlier Synthesis (GCOS), a training-time regularization framework.<n>The framework naturally transitions to conformal OOD inference, which translates uncertainty scores into statistically valid p-values.
  arXiv  Detail & Related papers  (2026-03-09T14:11:47Z)
• The Emergence of Lab-Driven Alignment Signatures: A Psychometric Framework for Auditing Latent Bias and Compounding Risk in Generative AI [0.0]
  This paper introduces a novel auditing framework to quantify latent trait estimation under ordinal uncertainty.<n>The research audits nine leading models across dimensions including Optimization Bias, Sycophancy, and Status-Quo Legitimization.
  arXiv  Detail & Related papers  (2026-02-19T06:56:01Z)
• The Stationarity Bias: Stratified Stress-Testing for Time-Series Imputation in Regulated Dynamical Systems [0.098314893665023]
  Time-series imputation benchmarks use random masking and shape-agnostic metrics.<n>We formalize this bias and propose a emphStratified Stress-Test that partitions evaluation into Stationary and Transient regimes.
  arXiv  Detail & Related papers  (2026-02-17T15:05:56Z)
• Causal Imitation Learning Under Measurement Error and Distribution Shift [6.038778620145853]
  We study offline imitation learning (IL) when part of the decision-relevant state is observed only through noisy measurements.<n>We propose a general framework for IL under measurement error, inspired by explicitly modeling the causal relationships among the variables.
  arXiv  Detail & Related papers  (2026-01-29T18:06:53Z)
• Reference-Specific Unlearning Metrics Can Hide the Truth: A Reality Check [60.77691669644931]
  We propose Functional Alignment for Distributional Equivalence (FADE), a novel metric that measures distributional similarity between unlearned and reference models.<n>We show that FADE captures functional alignment across the entire output distribution, providing a principled assessment of genuine unlearning.<n>These findings expose fundamental gaps in current evaluation practices and demonstrate that FADE provides a more robust foundation for developing and assessing truly effective unlearning methods.
  arXiv  Detail & Related papers  (2025-10-14T20:50:30Z)
• Reference-Free Rating of LLM Responses via Latent Information [53.463883683503106]
  We study the common practice of asking a judge model to assign Likert-scale scores to free-text responses.<n>We then propose and evaluate Latent Judges, which derive scalar ratings from internal model signals.<n>Across a broad suite of pairwise and single-rating benchmarks, latent methods match or surpass standard prompting.
  arXiv  Detail & Related papers  (2025-09-29T12:15:52Z)
• WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales [22.789611187514975]
  Methods for nonparametric sequential testing -- especially conformal test martingales (CTMs) and anytime-valid inference -- offer promising tools for this monitoring task.<n>Existing approaches are restricted to monitoring limited hypothesis classes or alarm criteria''
  arXiv  Detail & Related papers  (2025-05-07T17:53:47Z)
• Adaptive Scoring and Thresholding with Human Feedback for Robust Out-of-Distribution Detection [6.192472816262214]
  Machine Learning (ML) models are trained on in-distribution (ID) data but often encounter out-of-distribution (OOD) inputs during deployment.<n>Recent works have focused on designing scoring functions to quantify OOD uncertainty.<n>We propose a human-in-the-loop framework that emphsafely updates both scoring functions and thresholds on the fly based on real-world OOD inputs.
  arXiv  Detail & Related papers  (2025-05-05T00:25:14Z)
• TrustLoRA: Low-Rank Adaptation for Failure Detection under Out-of-distribution Data [62.22804234013273]
  We propose a simple failure detection framework to unify and facilitate classification with rejection under both covariate and semantic shifts.<n>Our key insight is that by separating and consolidating failure-specific reliability knowledge with low-rank adapters, we can enhance the failure detection ability effectively and flexibly.
  arXiv  Detail & Related papers  (2025-04-20T09:20:55Z)
• Selective Learning: Towards Robust Calibration with Dynamic Regularization [79.92633587914659]
  Miscalibration in deep learning refers to there is a discrepancy between the predicted confidence and performance. We introduce Dynamic Regularization (DReg) which aims to learn what should be learned during training thereby circumventing the confidence adjusting trade-off.
  arXiv  Detail & Related papers  (2024-02-13T11:25:20Z)
• When Does Confidence-Based Cascade Deferral Suffice? [69.28314307469381]
  Cascades are a classical strategy to enable inference cost to vary adaptively across samples. A deferral rule determines whether to invoke the next classifier in the sequence, or to terminate prediction. Despite being oblivious to the structure of the cascade, confidence-based deferral often works remarkably well in practice.
  arXiv  Detail & Related papers  (2023-07-06T04:13:57Z)
• Integrative conformal p-values for powerful out-of-distribution testing with labeled outliers [1.6371837018687636]
  This paper develops novel conformal methods to test whether a new observation was sampled from the same distribution as a reference set. The described methods can re-weight standard conformal p-values based on dependent side information from known out-of-distribution data. The solution can be implemented either through sample splitting or via a novel transductive cross-validation+ scheme.
  arXiv  Detail & Related papers  (2022-08-23T17:52:20Z)
• GenDICE: Generalized Offline Estimation of Stationary Values [108.17309783125398]
  We show that effective estimation can still be achieved in important applications. Our approach is based on estimating a ratio that corrects for the discrepancy between the stationary and empirical distributions. The resulting algorithm, GenDICE, is straightforward and effective.
  arXiv  Detail & Related papers  (2020-02-21T00:27:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.