Online Reward-Punishment Learning from Fixed-Channel Perceptual Event Streams without Environment Rewards

• URL: http://arxiv.org/abs/2606.18963v1
• Date: Wed, 17 Jun 2026 11:43:10 GMT
• Title: Online Reward-Punishment Learning from Fixed-Channel Perceptual Event Streams without Environment Rewards
• Abstract summary: OHIRL separates four roles: M_psi learns next-packet prediction, D_omega models residual dynamics, C_eta is a fixed internal post-transition trajectory.<n>C_eta uses a recovery-positive and persistence/growth-negative residual-regulation orientation.<n>A conditional error decomposition separates B_xi evidence-estimation error from residual policy-optimization error.
• Score: 3.1050763275397806
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study online reward-punishment learning when the environment provides no scalar reward or evaluative label. At each step the agent receives only a fixed-channel perceptual packet, and quantities such as pain, energy, contact, damage, or cognitive error are treated as perceptual dimensions whose valence must be inferred from transition consequences. OHIRL separates four roles: M_psi learns next-packet prediction, D_omega models residual dynamics, C_eta is a fixed internal post-transition trajectory evaluator, and B_xi learns to use the resulting value evidence for later policy updates and action scoring. C_eta uses a recovery-positive and persistence/growth-negative residual-regulation orientation; a coefficient-origin audit shows that equal-unit, raw-equal, and random monotone variants preserve more than 92% of the released top-action rankings, while sign inversion preserves 0%. The reward-free protocol exposes observation transitions while withholding environment rewards, delayed external evaluators, success labels, and action-goodness labels. A conditional error decomposition separates B_xi evidence-estimation error from residual policy-optimization error. In a 2x2-XOR packet task, medicine and chili acquire opposite value under visual XOR contexts, and the same pain or spice increase can be positive or negative depending on consequence structure; B_xi reaches 0.952 balanced reward-sign accuracy. In a full online-interleaved audit, M_psi reaches holdout R2=0.907, B_xi reaches 0.940 sign accuracy, and the policy reaches 0.979 optimal-action accuracy, while immediate packet scores, prediction-error rewards, shuffled targets, zero reward, and error-reduction controls collapse. Hidden-reward CartPole and Taxi controls, public-context no-leakage audits, and module-role ablations further test information boundaries and component necessity.

Related papers

• Semiparametric Inference for Conditional Shapley Feature Importance [0.0]
  This paper studies the conditional formulation, in which out-of-coalition features are integrated out under their true conditional distribution.<n>We propose a one-step estimator with K-fold cross-fitting and a U-statistic correction of the squared loss interval.<n>A Pinsker-type bound quantifies the bias from misspecifying the working copula class, while vine copulas keep conditional sampling tractable.
  arXiv  Detail & Related papers  (2026-09-09T15:19:24Z)
• Bayes-Optimal BER and AUC: Estimation and Evaluation of Estimators [47.001801756596926]
  A fundamental quantity in machine learning is the optimal performance achievable by any model on a given task.<n>Recent work has shown that the Bayes error, or equivalently the optimal accuracy, can be estimated from soft labels in binary classification.<n>We propose soft-label-based estimators for the optimal BER and AUC.
  arXiv  Detail & Related papers  (2026-09-02T08:49:23Z)
• Convex-Hull-Neighborhood Smooth Dual Generalization: Controlling Local Correction Propagation in Offline RL [7.803284875836737]
  bootstrapping can amplify estimation errors in offline reinforcement learning.<n>We propose Convex Hull Neighborhood Smooth Dual Generalization, which expresses the Bellman backup as an in-sample value target plus a CHN-local correction.<n> Experiments on Gym-MuJoCo and AntMaze show strong aggregate performance and stable value estimation.
  arXiv  Detail & Related papers  (2026-08-04T04:30:02Z)
• Abstention as an Action Can Kill Both the Reward Gradient and the KL Anchor: Collapse Law and Repair for Error-Penalized Reinforcement Learning [5.990509154718772]
  Error-penalized scoring rules ($+1$ for a correct answer, $$ for a wrong one, $0$ for abstaining) are increasingly prescribed against hallucination.<n>We prove that a KL-anchored learner can do the opposite.
  arXiv  Detail & Related papers  (2026-07-31T21:31:07Z)
• Counterfactual Shapley Credit Assignment [49.39649767581182]
  Credit Assignment Problem (CAP) is fundamental to developing efficient and explainable Reinforcement Learning (RL) agents.<n>We introduce Counterfactual Shapley Credit Assignment, a novel framework that attributes credit and blame via the Counterfactual Shapley ($$-value)<n>We derive a consistent estimator that computes $$$-values, enabling a new class of policy methods, $$-PPO, combined with Prioritized Trajectory Replay (PTR)
  arXiv  Detail & Related papers  (2026-07-18T23:28:20Z)
• From Checker to Forecaster: Code-Owned Evaluation of Model-Generated Strategic Routes Under Delayed Ground Truth [0.0]
  RouteCast instantiates the regime for model-generated typed strategic routes.<n>It produces a provisional forecast-ranking; later outcomes evaluate the forecast.<n>It does not establish prospective calibration, causal decision improvement, route-decomposition advantage, or cross-domain validity.
  arXiv  Detail & Related papers  (2026-07-13T00:36:48Z)
• Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures [0.0]
  Causal Agent Replay (CAR) models an agent run as a structural causal model.<n>It applies a do-operation to a step, and re-executes a trajectory forward under the same policy.<n>CAR is open source and runs on hosted or free local models.
  arXiv  Detail & Related papers  (2026-06-06T17:44:23Z)
• A Pre-Registered Causal Partition of Self-Consistency Elicitation and Reward Design in RLVR [1.2958054117511815]
  Reinforcement learning from rewards improves reasoning even when the reward signal is spurious.<n> Practitioners commonly interpret naive = acc(TRUE) - acc(R) as the reward-design effect.<n>We prove this estimand is systematically biased.
  arXiv  Detail & Related papers  (2026-06-04T09:35:54Z)
• Correction and Corruption: A Two-Rate View of Error Flow in LLM Protocols [51.56484100374058]
  We propose a paired-outcome measurement interface for auditing a single protocol step on exact-match tasks.<n>For each instance, the interface records a baseline correctness bit and a post-step correctness bit.<n>These rates predict accuracy changes and define a reusable empirical interface testable across seeds, mixtures, and pipelines.
  arXiv  Detail & Related papers  (2026-04-20T13:25:40Z)
• Conservative Offline Robot Policy Learning via Posterior-Transition Reweighting [36.23660195981511]
  Posterior-Transition Reweighting (PTR) is a reward-free and conservative post-training method.<n>It decides how much each training sample should influence the supervised update.<n>PTR reallocates credit according to how attributable each sample's post-action consequence is under the current representation.
  arXiv  Detail & Related papers  (2026-03-17T14:05:35Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• Will My Robot Achieve My Goals? Predicting the Probability that an MDP Policy Reaches a User-Specified Behavior Target [56.99669411766284]
  As an autonomous system performs a task, it should maintain a calibrated estimate of the probability that it will achieve the user's goal. This paper considers settings where the user's goal is specified as a target interval for a real-valued performance summary. We compute the probability estimates by inverting conformal prediction.
  arXiv  Detail & Related papers  (2022-11-29T18:41:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.