Geometry of Forgetting: Representation Flux in Continual Learning

• URL: http://arxiv.org/abs/2608.15854v1
• Date: Sun, 16 Aug 2026 16:59:03 GMT
• Title: Geometry of Forgetting: Representation Flux in Continual Learning
• Abstract summary: Catastrophic forgetting remains a fundamental obstacle to continual learning, where neural networks lose previously acquired knowledge while learning new tasks.<n>We investigate latent representation evolution during sequential learning and introduce representation flux, a geometric measure of sample-level representation displacement across training.<n>We show that representation flux is strongly associated with catastrophic forgetting across multiple benchmarks, with temporal analyses indicating that elevated flux can precede subsequent performance degradation.<n>We propose FlowLess-R, a representation-space regularization method that constrains replay representations relative to stored references while allowing continued learning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Catastrophic forgetting remains a fundamental obstacle to continual learning, where neural networks lose previously acquired knowledge while learning new tasks. Existing methods primarily mitigate forgetting through parameter regularization or experience replay, while the representation-space dynamics associated with forgetting remain less understood. We investigate latent representation evolution during sequential learning and introduce representation flux, a geometric measure of sample-level representation displacement across training. We show that representation flux is strongly associated with catastrophic forgetting across multiple benchmarks, with temporal analyses indicating that elevated flux can precede subsequent performance degradation. Representation displacement is also associated with confidence degradation, while complementary geometric properties provide additional information about sample-level forgetting. Motivated by these observations, we propose FlowLess-R, a representation-space regularization method that constrains replay representations relative to stored references while allowing continued learning. FlowLess-R is architecture-agnostic and integrates into replay-based methods through a representation-matching term. Experiments on SplitMNIST, SplitFashionMNIST, SplitCIFAR10, and SplitTinyImageNet show improved final average accuracy and reduced forgetting with ER, DER++, and ER-ACE. Our results identify representation flux as an informative geometric marker of forgetting and show that stabilizing latent representations provides a simple strategy for mitigating catastrophic forgetting.

Related papers

• Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning [41.371439825328544]
  Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting.<n>We develop a layer-wise information-theoretic framework that separates these effects at every depth.<n>Controlled and benchmark experiments confirm the predicted memory scaling, the interior funnel, and the alignment signal's link to forgetting.
  arXiv  Detail & Related papers  (2026-08-12T06:01:49Z)
• Adaptive Recurrent Message Passing for Test Time Computing on Graphs [80.56829418752513]
  We show that recurrent graph models can overcome the intrinsic mismatch between graph data and architectural designs.<n>We propose AdaR, an Adaptive Recurrent graph model, empowering flexible test-time computing on various downstream tasks without changing model parameters.
  arXiv  Detail & Related papers  (2026-06-21T12:14:11Z)
• Embedded Polygon Symbolic Transfer Entropy (EPSTE): A Geometric Token and Deep Learning Approach to Estimating Transfer Entropy in Neuroimaging Time Series [2.1575136240463597]
  This work introduces Embedded Polygon Transfer Entropy (EPSTE), a framework that reframes TE estimation as a learnable problem.<n>A recurrent neural network with attention-based multiple-instance learning is trained to predict surrogate-validated TE values from bags of symbolic temporal windows.<n>The results demonstrate that while local window-level predictions are noisy, aggregation across trials and channel pairs yields stable directed dependencies.
  arXiv  Detail & Related papers  (2026-06-19T21:13:56Z)
• Next-Acceleration-Scale Prediction for Autoregressive MRI Reconstruction [52.32112533846212]
  MRI reconstruction is an inherently ill-posed inverse problem, since incomplete measurements admit many plausible solutions.<n>We address this limitation by moving reconstruction to discrete multi-scale latent space and posing it as autoregressive next-acceleration-scale prediction.<n>We show that our approach delivers improved reconstruction performance across diverse sampling patterns under extreme undersampling.
  arXiv  Detail & Related papers  (2026-05-19T04:40:50Z)
• Monitoring Neural Training with Topology: A Footprint-Predictable Collapse Index [51.56484100374058]
  We propose an online, topology-aware monitor for evolving neural representations that couples Modular Morse Homology Maintenance (MMHM) with a composite Collapse Index (CI)<n>Instead of rebuilding complexes each epoch, we apply sparse edits at a fixed scale and maintain a discrete Morse matching, yielding fast, incremental updates.
  arXiv  Detail & Related papers  (2026-04-28T19:21:21Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• Squeezing More from the Stream : Learning Representation Online for Streaming Reinforcement Learning [14.799267729619428]
  In streaming Reinforcement Learning (RL), transitions are observed and discarded immediately after a single update.<n>We propose extending Self-Predictive Representations (SPR) to the streaming pipeline to maximize the utility of every observed frame.<n>We show that our method learns significantly richer representations, bridging the performance gap caused by the absence of a replay buffer.
  arXiv  Detail & Related papers  (2026-02-10T04:06:32Z)
• Disentangled Representation Learning via Flow Matching [48.12507436294143]
  Disentangled representation learning aims to capture the underlying explanatory factors of observed data.<n>Existing diffusion-based methods encourage factor independence via inductive biases, yet frequently lack strong semantic alignment.<n>We propose a flow matching-based framework for disentangled representation learning, which casts disentanglement as learning factor-conditioned flows in a compact latent space.
  arXiv  Detail & Related papers  (2026-02-05T02:14:36Z)
• Invariance on Manifolds: Understanding Robust Visual Representations for Place Recognition [19.200074425090595]
  We propose a Second-Order Geometric Statistics framework that inherently captures geometric stability without training.<n>Our approach introduces a training-free framework built upon fixed, pre-trained backbones, achieving strong zero-shot generalization without parameter updates.
  arXiv  Detail & Related papers  (2026-01-31T18:12:29Z)
• On the Importance of Feature Decorrelation for Unsupervised Representation Learning in Reinforcement Learning [23.876039876806182]
  unsupervised representation learning (URL) has improved the sample efficiency of Reinforcement Learning (RL) We propose a novel URL framework that causally predicts future states while increasing the dimension of the latent manifold. Our framework effectively learns predictive representations without collapse, which significantly improves the sample efficiency of state-of-the-art URL methods on the Atari 100k benchmark.
  arXiv  Detail & Related papers  (2023-06-09T02:47:21Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.