The Terminal Representation in Reinforcement Learning

• URL: http://arxiv.org/abs/2605.31289v1
• Date: Fri, 29 May 2026 13:24:28 GMT
• Title: The Terminal Representation in Reinforcement Learning
• Abstract summary: We introduce a structurally distinct formulation: the terminal representation (TR)<n>The TR encodes reward- trajectories similarly to the DR, but can be learned as a lower-dimensional abstractionity object.<n>We show the TR is embedded in the top DR eigenvector, allowing it to capture the same underlying knowledge without eigendecomposition.
• Score: 3.0846824529023387
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Representation learning is a powerful tool for spatio-temporal abstraction within reinforcement learning (RL). Two well established approaches are through the successor representation (SR) and the default representation (DR). The SR encodes states by the future trajectories they induce, capturing information flow decoupled from reward. The DR builds on this by weighting trajectories with reward, integrating credit-assignment structure into the representation. Eigenvectors of both representations have been used to support a range of downstream tasks -- including option discovery, reward shaping, transfer learning, and exploration. We introduce a structurally distinct formulation: the terminal representation (TR). The TR encodes reward-weighted trajectories similarly to the DR, but can be learned as a lower-dimensionality object, and can be used directly for the mentioned applications without eigenvector computations. Eigendecomposition also imposes the assumption of symmetric transition dynamics, which the TR can bypass. In this work we develop the theoretical foundations of the TR: its derivation, convergence of two learning algorithms, its use for zero-shot compositionality, and equivalences between alternative reward formulations. We further show the TR is embedded in the top DR eigenvector, allowing it to capture the same underlying knowledge without eigendecomposition. Additionally, we provide empirical evidence of the TR as a viable alternative to existing representations in subsidiary applications, while requiring less computational overhead to learn, store, and use.

Related papers

• REFINE: Trajectory Representation Learning via Closed-Loop Transcription -- Extended Version [22.872723951282282]
  REFINE is a simple yet effective lEarning Framework vIa closed-loop trajector rEfinement for trajectory data.<n>We provide a control-theoretic analysis that establishes guarantees for the proposed closed-loop optimization.<n>Experiments on four real-world datasets demonstrate that REFINE consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-09-07T08:26:49Z)
• TRACE: Task-Adaptive Reasoning and Representation Learning for Universal Multimodal Retrieval [35.86480813138274]
  Universal Multimodal Retrieval requires unified embedding models capable of interpreting diverse user intents.<n>We introduce TRACE (Task-adaptive Reasoning And Embeddings)<n>TRACE unifies generative reasoning with discriminative representation learning.
  arXiv  Detail & Related papers  (2026-03-03T12:36:39Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• DROGO: Default Representation Objective via Graph Optimization in Reinforcement Learning [14.390952817249746]
  We derive an objective for directly approximating the principal eigenvector of the default representation with a neural network.<n>We empirically demonstrate the effectiveness of the objective in a number of environments, and apply the learned eigenvectors for reward shaping.
  arXiv  Detail & Related papers  (2026-01-30T23:45:45Z)
• From Verifiable Dot to Reward Chain: Harnessing Verifiable Reference-based Rewards for Reinforcement Learning of Open-ended Generation [52.62655622099456]
  We propose reinforcement learning with verifiable reference-based rewards (RLVRR)<n>Instead of checking the final answer, RLVRR extracts an ordered linguistic signal from high-quality references (i.e., reward chain)<n>In this way, RLVRR decomposes rewards into two dimensions: content, which preserves deterministic core concepts, and style, which evaluates adherence to stylistic properties.
  arXiv  Detail & Related papers  (2026-01-26T14:39:58Z)
• Knowledge-Informed Neural Network for Complex-Valued SAR Image Recognition [51.03674130115878]
  We introduce the Knowledge-Informed Neural Network (KINN), a lightweight framework built upon a novel "compression-aggregation-compression" architecture.<n>KINN establishes a state-of-the-art in parameter-efficient recognition, offering exceptional generalization in data-scarce and out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2025-10-23T07:12:26Z)
• From Novelty to Imitation: Self-Distilled Rewards for Offline Reinforcement Learning [7.559920170287638]
  Offline Reinforcement Learning (RL) aims to learn effective policies from a static dataset without requiring further agent-environment interactions.<n>We propose ReLOAD, a novel reward annotation framework for offline RL.<n>Our approach adapts Random Network Distillation (RND) to generate intrinsic rewards from expert demonstrations.
  arXiv  Detail & Related papers  (2025-07-17T06:16:06Z)
• Kolmogorov GAM Networks are all you need! [0.6906005491572398]
  Kolmogorov GAM networks are shown to be an efficient architecture for training and inference.<n>They are an additive model with an embedding that is independent of the function of interest.
  arXiv  Detail & Related papers  (2025-01-01T02:46:00Z)
• Probabilistic Abduction for Visual Abstract Reasoning via Learning Rules in Vector-symbolic Architectures [22.12114509953737]
  Abstract reasoning is a cornerstone of human intelligence, and replicating it with artificial intelligence (AI) presents an ongoing challenge. This study focuses on efficiently solving Raven's progressive matrices (RPM), a visual test for assessing abstract reasoning abilities. Instead of hard-coding the rule formulations associated with RPMs, our approach can learn the VSA rule formulations with just one pass through the training data.
  arXiv  Detail & Related papers  (2024-01-29T10:17:18Z)
• REBAR: Retrieval-Based Reconstruction for Time-series Contrastive Learning [64.08293076551601]
  We propose a novel method of using a learned measure for identifying positive pairs. Our Retrieval-Based Reconstruction measure measures the similarity between two sequences. We show that the REBAR error is a predictor of mutual class membership.
  arXiv  Detail & Related papers  (2023-11-01T13:44:45Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPs [53.710405006523274]
  This work focuses on the representation learning question: how can we learn such features? Under the assumption that the underlying (unknown) dynamics correspond to a low rank transition matrix, we show how the representation learning question is related to a particular non-linear matrix decomposition problem. We develop FLAMBE, which engages in exploration and representation learning for provably efficient RL in low rank transition models.
  arXiv  Detail & Related papers  (2020-06-18T19:11:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.