Exploratory Experience Shapes the Geometry of Predictive Representations

• URL: http://arxiv.org/abs/2605.27929v1
• Date: Wed, 27 May 2026 04:01:33 GMT
• Title: Exploratory Experience Shapes the Geometry of Predictive Representations
• Abstract summary: We build an online learning agent in a tree-like maze with a controllable parameter regulating the balance between exploratory and exploitative regimes.<n>The model predicts both future maze states and reward probability, allowing the agent to select actions either by expected information gain during exploration or by predicted reward during exploitation.<n>We show that the resulting internal predictive representations depend strongly on the agent's behavioral regime.
• Score: 0.6999740786886536
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Active sensing links behavior and learning through an action-perception loop: actions determine the observations used to update internal predictive models of perception, which subsequently guide the next actions. Predictive-coding frameworks provide a natural way to model this process, since internal representations are continuously updated to predict future observations. Here, we ask how exploratory and exploitative behavioral strategies shape these internal predictive representations. We build an online learning agent in a tree-like maze with a controllable parameter regulating the balance between exploratory and exploitative regimes. The agent updates a predictive-coding-based perception model from experience generated by its own behavior. The model predicts both future maze states and reward probability, allowing the agent to select actions either by expected information gain during exploration or by predicted reward during exploitation. We show that the resulting internal predictive representations depend strongly on the agent's behavioral regime. Exploratory agents develop representations that are more spatially organized and better preserve the structure of maze transitions in latent space. In contrast, exploitative agents learn less organized representations. We then train this predictive model on natural trajectories of water-deprived mice navigating the same maze and compare the resulting representations with those learned from agent trajectories. More exploratory mice show representational geometries that closely match those of exploratory agents, whereas mice with more restricted visitation patterns resemble reward-driven, exploitative agents. Together, these findings suggest that exploration enables predictive models to form generalized internal representations by organizing latent space around both spatial location and transition context in artificial agents and animals.

Related papers

• How to Interpret Agent Behavior [56.59836196946289]
  We introduce ACT*ONOMY, a taxonomy for describing and analyzing agent behavior at runtime.<n>By providing a shared vocabulary, ACT*ONOMY helps researchers, agent designers, and end users interpret agent behavior more consistently.
  arXiv  Detail & Related papers  (2026-05-13T14:52:40Z)
• LatentPilot: Scene-Aware Vision-and-Language Navigation by Dreaming Ahead with Latent Visual Reasoning [51.969318585152116]
  LatentPilot exploits future observations during training as a valuable data source to learn action-conditioned visual dynamics.<n>We propose a flywheel-style training mechanism that iteratively collects on-policy trajectories and retrains the model to better match the agent's behavior distribution.<n> Experiments on R2R-CE, RxR-CE, and R2R-PE benchmarks achieve new SOTA results, and real-robot tests across diverse environments demonstrate LatentPilot's superior understanding of environment-action dynamics in scene.
  arXiv  Detail & Related papers  (2026-03-31T02:21:59Z)
• Future-Interactions-Aware Trajectory Prediction via Braid Theory [4.331354753582517]
  We propose a novel auxiliary task, braid prediction, done in parallel with the trajectory prediction task.<n>We show how the braid prediction task infuses the model with future awareness intention, leading to more accurate joint predictions.
  arXiv  Detail & Related papers  (2026-03-23T14:38:15Z)
• Integrating Specialized and Generic Agent Motion Prediction with Dynamic Occupancy Grid Maps [3.3894571022475066]
  We propose a unified framework to simultaneously predict future occupancy state grids, vehicle grids, and scene flow grids.<n>Our approach is centered on a tailored, interdependent loss function that captures inter-grid dependencies and enables diverse future predictions.<n> Evaluations on real-world nuScenes and Woven Planet datasets demonstrate superior prediction performances for dynamic vehicles and generic dynamic scene elements.
  arXiv  Detail & Related papers  (2026-02-08T12:13:06Z)
• Current Agents Fail to Leverage World Model as Tool for Foresight [61.82522354207919]
  Generative world models offer a promising remedy: agents could use them to foresee outcomes before acting.<n>This paper empirically examines whether current agents can leverage such world models as tools to enhance their cognition.
  arXiv  Detail & Related papers  (2026-01-07T13:15:23Z)
• TRACE: A Self-Improving Framework for Robot Behavior Forecasting with Vision-Language Models [1.3408365072149797]
  Predicting the near-term behavior of a reactive agent is crucial in many robotic scenarios.<n>We present TRACE, an inference framework that couples tree-of-thought generation with domain-aware feedback.<n>We validate TRACE on both ground-vehicle simulations and real-world marine autonomous surface vehicles.
  arXiv  Detail & Related papers  (2025-03-02T06:58:02Z)
• EgoAgent: A Joint Predictive Agent Model in Egocentric Worlds [119.02266432167085]
  We propose EgoAgent, a unified agent model that simultaneously learns to represent, predict, and act within a single transformer.<n>EgoAgent explicitly models the causal and temporal dependencies among these abilities by formulating the task as an interleaved sequence of states and actions.<n> Comprehensive evaluations of EgoAgent on representative tasks such as image classification, egocentric future state prediction, and 3D human motion prediction demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2025-02-09T11:28:57Z)
• Behavior-Inspired Neural Networks for Relational Inference [6.855586051080836]
  We study how interactions between agents govern how dynamical systems evolve in space and time.<n>Recent works learn to categorize relationships between agents based on observations of their physical behavior.<n>We introduce a level of abstraction between the observable behavior of agents and the latent categories that determine their behavior.
  arXiv  Detail & Related papers  (2024-06-20T21:36:54Z)
• Finding Useful Predictions by Meta-gradient Descent to Improve Decision-making [1.384055225262046]
  We focus on predictions expressed as General Value Functions: temporally extended estimates of the accumulation of a future signal. One challenge is determining from the infinitely many predictions that the agent could possibly make which might support decision-making. By learning, rather than manually specifying these predictions, we enable the agent to identify useful predictions in a self-supervised manner.
  arXiv  Detail & Related papers  (2021-11-18T20:17:07Z)
• You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction [52.442129609979794]
  Recent deep learning approaches for trajectory prediction show promising performance. It remains unclear which features such black-box models actually learn to use for making predictions. This paper proposes a procedure that quantifies the contributions of different cues to model performance.
  arXiv  Detail & Related papers  (2021-10-11T14:24:15Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.