Breaking Feedback-Blindness: Utility-Augmented Transformer for Sequential Decision Making

• URL: http://arxiv.org/abs/2607.18910v1
• Date: Tue, 21 Jul 2026 09:52:23 GMT
• Title: Breaking Feedback-Blindness: Utility-Augmented Transformer for Sequential Decision Making
• Abstract summary: Utility-Augmented Transformer is a new feedback-conditioned retrieval attention architecture.<n>We show that UAT consistently improves performance over observation-only, test-time adaptation, and input-level feedback baselines.
• Score: 0.16385815610837165
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sequential decision making in non-stationary and partially observable environments requires rapid adaptation to latent regime changes. However, existing Transformer decision models face a structural bottleneck in the retrieval mechanism: even when reward is used for training or exposed as an input token, attention retrieval remains primarily driven by observation-derived similarity. We formalize this limitation as feedback-blind retrieval, and formally show that, on feedback-informative tasks, observation-equivalent histories with different action-reward outcomes cannot be distinguished by any observation-only attention, resulting in suboptimal choice. To address this mismatch, we propose the Utility-Augmented Transformer (UAT), a new feedback-conditioned retrieval attention architecture in which a compact utility state modulates the query, key, and value projections, allowing action-reward history to directly alter context retrieval during the forward pass. UAT also enjoys an exact zero-gate degradation property that recovers the Vanilla Transformer when feedback is uninformative. Under finite-horizon compactness and Lipschitz assumptions, we prove that UAT strictly enlarges the observation-only Transformer class and can uniformly approximate feedback-dependent decision maps. Across four non-stationary benchmarks: synthetic navigation with hidden goal shifts, non-stationary sepsis treatment, cross-market portfolio allocation, and delayed-feedback recommendation, UAT consistently improves performance over observation-only, test-time adaptation, and input-level feedback baselines, with particularly large gains in noisier regimes that require stronger adaptation.

Related papers

• ResLRP: The Role of Residual Cancellation in Attribution Instability in Vision Transformers [58.411900827911346]
  We show that the missing ingredient is the treatment of residual connections.<n>We introduce Residual-aware Layer-wise Relevance Propagation (ResLRP)<n>ResLRP is a simple extension of LRP whose propagation rules explicitly account for cancellations in residual branches.
  arXiv  Detail & Related papers  (2026-09-15T13:17:30Z)
• PL-SCEA: Reconfiguring Pretrained Attention for Few-Shot Industrial Anomaly Detection [2.4104933690855153]
  Vision Foundation Models (VFMs) provide transferable patch representations for few-shot industrial anomaly detection.<n>We investigate the hypothesis that the attention of a frozen VFM can be reconfigured as a task-relevant component of anomaly detection.<n>We instantiate this idea with Power-Law Self-Correlation Enhanced Attention (PL-SCEA), which retains the semantic context of pretrained query-key attention.
  arXiv  Detail & Related papers  (2026-09-03T10:54:05Z)
• HERO: Hindsight-Enhanced Reflection from Environment Observations for Agentic Self-Distillation [50.53459634301361]
  HERO is a hindsight-enhanced self-distillation framework that uses next environment observations as locally aligned feedback.<n> HERO improves task success and reduces unnecessary turns over environment-feedback-only self-distillation and GRPO.
  arXiv  Detail & Related papers  (2026-06-10T01:35:34Z)
• Factorize to Generalize: Retrieval-Guided Invariant-Dynamic Decomposition for Time Series Forecasting [64.2640656619583]
  Time series foundation models (TSFMs) have recently achieved strong zero-shot forecasting performance through large-scale pretraining and retrieval-augmented prediction.<n>We propose a Retrieval-guided Invariant-Dynamic DEcomposition framework for time series forecasting.
  arXiv  Detail & Related papers  (2026-05-24T07:31:55Z)
• MoASE++: Mixture of Activation Sparsity Experts with Domain-Adaptive On-policy Distillation for Continual Test Time Adaptation [65.068801413044]
  Continual test-time adaptation adapts a source-pretrained model to non-stationary, unlabeled target streams.<n>We introduce MoASE, a mixture-of-experts that disentangles domain-agnostic structure from domain-specific texture.
  arXiv  Detail & Related papers  (2026-05-18T01:52:12Z)
• Temporal Structure Matters for Efficient Test-Time Adaptation in Wearable Human Activity Recognition [0.20391237204597365]
  Test-time adaptation (TTA) mitigates performance degradation under real-world cross-user distribution shifts.<n>Existing methods largely inherit assumptions from vision tasks and underexploit the inherent inter-window temporal structure in WHAR streams.<n>We propose SIGHT, a lightweight and backpropagation-free TTA framework for WHAR, enabling real-time edge deployment.
  arXiv  Detail & Related papers  (2026-05-06T08:06:27Z)
• FACTOR: Counterfactual Training-Free Test-Time Adaptation for Open-Vocabulary Object Detection [63.91351553178842]
  FACTOR is a training-free Test-time adaptation for Open-vocabulary object detection.<n>It quantifies attribute sensitivity, semantic relevance, and prediction variation to selectively suppress attribute-dependent predictions.<n>Experiments on PASCAL-C, COCO-C, and FoggyCityscapes show that FACTOR consistently outperforms prior TTA methods.
  arXiv  Detail & Related papers  (2026-05-05T02:31:18Z)
• FIRCE: A Framework for Intrusion Response and Conformal Evaluation [3.3271517946886924]
  FIRCE is a framework for Intrusion Response and Conformal Evaluation.<n>It supports conformal evaluation-based uncertainty quantification and drift detection.<n>We demonstrate FIRCE's ability to detect distributional shifts and trigger model retraining.
  arXiv  Detail & Related papers  (2026-05-03T16:48:03Z)
• Feedback Adaptation for Retrieval-Augmented Generation [36.004563021329965]
  We introduce feedback adaptation as a problem setting for RAG systems.<n>We show that training-based approaches exhibit a trade-off between delayed correction and reliable adaptation.<n>Our results highlight feedback adaptation as a previously overlooked dimension of RAG system behavior in interactive settings.
  arXiv  Detail & Related papers  (2026-04-08T03:47:48Z)
• Promptable Anomaly Segmentation with SAM Through Self-Perception Tuning [63.55145330447408]
  We propose a novel textbfSelf-textbfPerceptinon textbfTuning (textbfSPT) method for anomaly segmentation.<n>The SPT method incorporates a self-drafting tuning strategy, which generates an initial coarse draft of the anomaly mask, followed by a refinement process.
  arXiv  Detail & Related papers  (2024-11-26T08:33:25Z)
• PseudoNeg-MAE: Self-Supervised Point Cloud Learning using Conditional Pseudo-Negative Embeddings [55.55445978692678]
  PseudoNeg-MAE enhances global feature representation of point cloud masked autoencoders by making them both discriminative and sensitive to transformations.<n>We propose a novel loss that explicitly penalizes invariant collapse, enabling the network to capture richer transformation cues while preserving discriminative representations.
  arXiv  Detail & Related papers  (2024-09-24T07:57:21Z)
• Action Quality Assessment with Temporal Parsing Transformer [84.1272079121699]
  Action Quality Assessment (AQA) is important for action understanding and resolving the task poses unique challenges due to subtle visual differences. We propose a temporal parsing transformer to decompose the holistic feature into temporal part-level representations. Our proposed method outperforms prior work on three public AQA benchmarks by a considerable margin.
  arXiv  Detail & Related papers  (2022-07-19T13:29:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.