Notes on the Reward Representation of Posterior Updates

• URL: http://arxiv.org/abs/2602.02912v1
• Date: Mon, 02 Feb 2026 23:37:39 GMT
• Title: Notes on the Reward Representation of Posterior Updates
• Abstract summary: We study the case where a KL-regularized soft update is exactly a Bayesian posterior inside a single fixed probabilistic model.<n>In this regime, behavioral change is driven only by evidence carried by that channel.<n> posterior updates determine the relative, context-dependent incentive signal that shifts behavior, but they do not uniquely determine absolute rewards.
• Score: 1.0247773757446124
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many ideas in modern control and reinforcement learning treat decision-making as inference: start from a baseline distribution and update it when a signal arrives. We ask when this can be made literal rather than metaphorical. We study the special case where a KL-regularized soft update is exactly a Bayesian posterior inside a single fixed probabilistic model, so the update variable is a genuine channel through which information is transmitted. In this regime, behavioral change is driven only by evidence carried by that channel: the update must be explainable as an evidence reweighing of the baseline. This yields a sharp identification result: posterior updates determine the relative, context-dependent incentive signal that shifts behavior, but they do not uniquely determine absolute rewards, which remain ambiguous up to context-specific baselines. Requiring one reusable continuation value across different update directions adds a further coherence constraint linking the reward descriptions associated with different conditioning orders.

Related papers

• A Composable CRDT Layer for Byzantine-Resilient Deterministic Reconstruction [0.40611352512781873]
  Conflict-free replicated data types (CRDTs) ensure Strong Even Consistency without coordination.<n>We address this problem through deterministic state reconstruction.<n>We show that its reconstruction model guarantees convergence even under arbitrary update injection.
  arXiv  Detail & Related papers  (2026-06-17T11:49:13Z)
• Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift [17.611062308867275]
  We show that distribution shift rotates the subspace that the model actively uses, misaligning the explainer's dictionary trained on in-distribution (ID) activations.<n>We formalize this misalignment as the faithfulness gap, a geometric distance between the ID dictionary and the OOD-active subspace.<n>We propose the Geometry-Adaptive Explainer (GAE), which realigns the explainer's dictionary with the OOD-active subspace while preserving the original feature structure.
  arXiv  Detail & Related papers  (2026-05-21T00:46:01Z)
• Trust the Batch, On- or Off-Policy: Adaptive Policy Optimization for RL Post-Training [50.86545293331458]
  Reinforcement learning is structurally harder than supervised learning because the policy changes the data distribution it learns from.<n>We present a simple yet effective batch-adaptive objective that replaces fixed clipping with the normalized effective sample size of the policy ratios.
  arXiv  Detail & Related papers  (2026-05-12T16:44:47Z)
• Tree of Concepts: Interpretable Continual Learners in Non-Stationary Clinical Domains [2.688900425133985]
  Continual learning aims to update models under distribution shift without forgetting, yet many high-stakes deployments, such as healthcare, also require interpretability.<n>We propose Tree of Concepts, an interpretable continual learning framework that uses a shallow decision tree to define a fixed, rule-based concept interface.<n>Our results suggest that structured concept interfaces can support continual adaptation while preserving a consistent audit interface in non-stationary, high-stakes domains.
  arXiv  Detail & Related papers  (2026-04-18T17:58:37Z)
• FILT3R: Latent State Adaptive Kalman Filter for Streaming 3D Reconstruction [51.56484100374058]
  Streaming 3D reconstruction maintains a persistent latent state that is updated online from incoming frames.<n>FILT3R casts recurrent state updates as state estimation in token space.<n>Code will be released at https://github.com/jinotter3/FILT3R.
  arXiv  Detail & Related papers  (2026-03-19T04:56:36Z)
• Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation [58.3773038915023]
  Continual learning aims to adapt pre-trained models to sequential tasks without forgetting previously acquired knowledge.<n>Most existing approaches treat continual learning as avoiding interference with past updates, rather than considering what properties make the current task-specific update naturally preserve previously acquired knowledge.<n>We address this problem using a projected first-order method compatible with standard deep-dots used in vision-language models.
  arXiv  Detail & Related papers  (2026-01-31T13:27:02Z)
• Explanation Multiplicity in SHAP: Characterization and Assessment [28.413883186555438]
  Post-hoc explanations are widely used to justify, contest, and review automated decisions in high-stakes domains such as lending, employment, and healthcare.<n>In practice, however, SHAP explanations can differ substantially across repeated runs, even when the individual, prediction task, and trained model are held fixed.<n>We conceptualize and name this phenomenon explanation multiplicity: the existence of multiple, internally valid but substantively different explanations for the same decision.
  arXiv  Detail & Related papers  (2026-01-19T02:01:18Z)
• Refinement Provenance Inference: Detecting LLM-Refined Training Prompts from Model Behavior [58.751981587234916]
  This paper formalizes the Refinement Provenance Inference (RPI) audit task as Refinement Provenance Inference (RPI)<n>We propose RePro, a logit-based framework that fuses teacher-forced likelihood features with logit-ranking signals.<n>During training, RePro learns a transferable representation via shadow fine-tuning, and uses a lightweight linear head to infer provenance on unseen victims without training-data access.
  arXiv  Detail & Related papers  (2026-01-05T10:16:41Z)
• InTAct: Interval-based Task Activation Consolidation for Continual Learning [8.228364777296195]
  InTAct preserves functional behavior in shared layers without freezing parameters or storing past data.<n>In doing so, InTAct stabilizes the functional role of important neurons rather than directly restricting parameter values.<n>Across diverse domain-incremental benchmarks, InTAct consistently reduces representation drift and improves performance.
  arXiv  Detail & Related papers  (2025-11-21T17:36:12Z)
• Geometric-Disentangelment Unlearning [106.99160454669902]
  gradient ascent on forget samples often harms retained knowledge.<n>We propose the Geometric-disment Unlearning (GU) that decomposes any candidate forget gradient update into tangential and normal components to retain space and executes only the normal component.<n>Our method is plug-and-play and can be attached to existing gradient-based unlearning procedures to mitigate side effects.
  arXiv  Detail & Related papers  (2025-11-21T09:58:25Z)
• The Mirror Loop: Recursive Non-Convergence in Generative Reasoning Systems [0.0]
  Recursive self-evaluation without external feedback often yields reformulation rather than progress.<n>We study 144 reasoning sequences across three models (OpenAI GPT-4o-mini, Anthropic Claude 3 Haiku, and Google Gemini 2.0 Flash) and four task families (arithmetic, code, explanation, reflection)<n>We interpret this as evidence for a structural limit on self-correction in generative reasoning.
  arXiv  Detail & Related papers  (2025-10-23T07:53:26Z)
• STABLE: Gated Continual Learning for Large Language Models [0.0]
  STABLE is a gated continual self editing framework that constrains forgetting during sequential updates.<n>Each candidate edit is evaluated against a stability budget using one of three metrics.<n>Experiments on the Qwen-2.5-7B model show that gating effectively mitigates forgetting while preserving adaptability.
  arXiv  Detail & Related papers  (2025-10-17T16:14:05Z)
• Value Flows [90.1510269525399]
  This paper uses modern, flexible flow-based models to estimate the full future return distributions.<n>Building upon the learned flow models, we estimate the return uncertainty of distinct states using a new flow derivative ODE.<n>Experiments on $37$ state-based and $25$ image-based benchmark tasks demonstrate that Value Flows achieves a $1.3times$ improvement on average in success rates.
  arXiv  Detail & Related papers  (2025-10-09T00:57:40Z)
• Generic Temporal Reasoning with Differential Analysis and Explanation [61.96034987217583]
  We introduce a novel task named TODAY that bridges the gap with temporal differential analysis. TODAY evaluates whether systems can correctly understand the effect of incremental changes. We show that TODAY's supervision style and explanation annotations can be used in joint learning.
  arXiv  Detail & Related papers  (2022-12-20T17:40:03Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• Optimal Change-Point Detection with Training Sequences in the Large and Moderate Deviations Regimes [72.68201611113673]
  This paper investigates a novel offline change-point detection problem from an information-theoretic perspective. We assume that the knowledge of the underlying pre- and post-change distributions are not known and can only be learned from the training sequences which are available.
  arXiv  Detail & Related papers  (2020-03-13T23:39:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.