Analysis of the Memorization and Generalization Capabilities of AI Agents: Are Continual Learners Robust?

• URL: http://arxiv.org/abs/2309.10149v2
• Date: Wed, 10 Jan 2024 16:07:12 GMT
• Title: Analysis of the Memorization and Generalization Capabilities of AI Agents: Are Continual Learners Robust?
• Authors: Minsu Kim and Walid Saad
• Abstract summary: In continual learning (CL), an AI agent learns from non-stationary data streams under dynamic environments. In this paper, a novel CL framework is proposed to achieve robust generalization to dynamic environments while retaining past knowledge. The generalization and memorization performance of the proposed framework are theoretically analyzed.
• Score: 91.682459306359
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In continual learning (CL), an AI agent (e.g., autonomous vehicles or robotics) learns from non-stationary data streams under dynamic environments. For the practical deployment of such applications, it is important to guarantee robustness to unseen environments while maintaining past experiences. In this paper, a novel CL framework is proposed to achieve robust generalization to dynamic environments while retaining past knowledge. The considered CL agent uses a capacity-limited memory to save previously observed environmental information to mitigate forgetting issues. Then, data points are sampled from the memory to estimate the distribution of risks over environmental change so as to obtain predictors that are robust with unseen changes. The generalization and memorization performance of the proposed framework are theoretically analyzed. This analysis showcases the tradeoff between memorization and generalization with the memory size. Experiments show that the proposed algorithm outperforms memory-based CL baselines across all environments while significantly improving the generalization performance on unseen target environments.

Related papers

• AMemGym: Interactive Memory Benchmarking for Assistants in Long-Horizon Conversations [61.6579785305668]
  AMemGym is an interactive environment enabling on-policy evaluation and optimization for memory-driven personalization.<n>Our framework provides a scalable, diagnostically rich environment for advancing memory capabilities in conversational agents.
  arXiv  Detail & Related papers  (2026-03-02T15:15:11Z)
• GLOVE: Global Verifier for LLM Memory-Environment Realignment [15.456830820378656]
  We propose a framework that introduces a new design dimension for Large Language Model memory systems by establishing a relative notion of truth.<n>GLOVE enables memory-environment realignment by verifying and updating memory without access to ground-truth supervision or strong reliance on model introspection.<n>Our results show that GLOVE substantially improves agent success rates, suggesting a robust pathway to cognitive agents capable of self-evolving.
  arXiv  Detail & Related papers  (2026-01-27T06:32:05Z)
• Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey [211.01908189012184]
  Memory, with hundreds of papers released this year, emerges as the critical solution to fill the utility gap.<n>We provide a unified view of foundation agent memory along three dimensions.<n>We then analyze how memory is instantiated and operated under different agent topologies.
  arXiv  Detail & Related papers  (2026-01-14T07:38:38Z)
• CREAM: Continual Retrieval on Dynamic Streaming Corpora with Adaptive Soft Memory [19.64051996386645]
  CREAM is a self-supervised framework for memory-based continual retrieval.<n>It adapts to both seen and unseen topics in an unsupervised setting.<n> Experiments on two benchmark datasets demonstrate that CREAM exhibits superior adaptability and retrieval accuracy.
  arXiv  Detail & Related papers  (2026-01-06T04:47:49Z)
• Interpretable Hybrid Deep Q-Learning Framework for IoT-Based Food Spoilage Prediction with Synthetic Data Generation and Hardware Validation [0.5417521241272645]
  The need for an intelligent, real-time spoilage prediction system has become critical in modern IoT-driven food supply chains.<n>We propose a hybrid reinforcement learning framework integrating Long Short-Term Memory (LSTM) and Recurrent Neural Networks (RNN) for enhanced spoilage prediction.
  arXiv  Detail & Related papers  (2025-12-22T12:59:48Z)
• Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory [89.65731902036669]
  Evo-Memory is a streaming benchmark and framework for evaluating self-evolving memory in large language model (LLM) agents.<n>We evaluate over ten representative memory modules and evaluate them across 10 diverse multi-turn goal-oriented and single-turn reasoning and QA datasets.
  arXiv  Detail & Related papers  (2025-11-25T21:08:07Z)
• Learning from Supervision with Semantic and Episodic Memory: A Reflective Approach to Agent Adaptation [11.819481846962447]
  We investigate how agents built on pretrained large language models can learn target classification functions from labeled examples without parameter updates.<n>Our framework uses episodic memory to store instance-level critiques and distill these into reusable, task-level guidance.<n>Our findings highlight the promise of memory-driven, reflective learning for building more adaptive and interpretable LLM agents.
  arXiv  Detail & Related papers  (2025-10-22T17:58:03Z)
• EvoEmpirBench: Dynamic Spatial Reasoning with Agent-ExpVer [5.855255212938064]
  We introduce two dynamic spatial benchmarks that evaluate models' abilities in spatial understanding and adaptive planning.<n>Experiments show that our benchmarks reveal key limitations of mainstream models in dynamic spatial reasoning and long-term memory.
  arXiv  Detail & Related papers  (2025-09-16T06:21:38Z)
• Quantifying Memory Utilization with Effective State-Size [73.52115209375343]
  We develop a measure of textitmemory utilization' This metric is tailored to the fundamental class of systems with textitinput-invariant and textitinput-varying linear operators
  arXiv  Detail & Related papers  (2025-04-28T08:12:30Z)
• Counterfactual experience augmented off-policy reinforcement learning [9.77739016575541]
  CEA builds efficient inference model and enhances representativeness of learning data. Uses variational autoencoders to model the dynamic patterns of state transitions. Builds a complete counterfactual experience to alleviate the out-of-distribution problem of the learning data.
  arXiv  Detail & Related papers  (2025-03-18T02:32:50Z)
• A General Close-loop Predictive Coding Framework for Auditory Working Memory [4.7368661961661775]
  We propose a general framework based on a close-loop predictive coding paradigm to perform short auditory signal memory tasks. The framework is evaluated on two widely used benchmark datasets for environmental sound and speech.
  arXiv  Detail & Related papers  (2025-03-16T13:57:37Z)
• C$^{2}$INet: Realizing Incremental Trajectory Prediction with Prior-Aware Continual Causal Intervention [10.189508227447401]
  Trajectory prediction for multi-agents in complex scenarios is crucial for applications like autonomous driving. Existing methods often overlook environmental biases, which leads to poor generalization. We propose the Continual Causal Intervention (C$2$INet) method for generalizable multi-agent trajectory prediction.
  arXiv  Detail & Related papers  (2024-11-19T08:01:20Z)
• DUEL: Duplicate Elimination on Active Memory for Self-Supervised Class-Imbalanced Learning [19.717868805172323]
  We propose an active data filtering process during self-supervised pre-training in our novel framework, Duplicate Elimination (DUEL) This framework integrates an active memory inspired by human working memory and introduces distinctiveness information, which measures the diversity of the data in the memory. The DUEL policy, which replaces the most duplicated data with new samples, aims to enhance the distinctiveness information in the memory and thereby mitigate class imbalances.
  arXiv  Detail & Related papers  (2024-02-14T06:09:36Z)
• Discovering and Reasoning of Causality in the Hidden World with Large Language Models [109.62442253177376]
  We develop a new framework termed Causal representatiOn AssistanT (COAT) to propose useful measured variables for causal discovery.<n>Instead of directly inferring causality with Large language models (LLMs), COAT constructs feedback from intermediate causal discovery results to LLMs to refine the proposed variables.
  arXiv  Detail & Related papers  (2024-02-06T12:18:54Z)
• An Adaptive Framework for Generalizing Network Traffic Prediction towards Uncertain Environments [51.99765487172328]
  We have developed a new framework using time-series analysis for dynamically assigning mobile network traffic prediction models. Our framework employs learned behaviors, outperforming any single model with over a 50% improvement relative to current studies.
  arXiv  Detail & Related papers  (2023-11-30T18:58:38Z)
• Generalization Across Observation Shifts in Reinforcement Learning [13.136140831757189]
  We extend the bisimulation framework to account for context dependent observation shifts. Specifically, we focus on the simulator based learning setting and use alternate observations to learn a representation space. This allows us to deploy the agent to varying observation settings during test time and generalize to unseen scenarios.
  arXiv  Detail & Related papers  (2023-06-07T16:49:03Z)
• AACC: Asymmetric Actor-Critic in Contextual Reinforcement Learning [13.167123175701802]
  This paper formalizes the task of adapting to changing environmental dynamics in Reinforcement Learning (RL) We then propose the Asymmetric Actor-Critic in Contextual RL (AACC) as an end-to-end actor-critic method to deal with such generalization tasks. We demonstrate the essential improvements in the performance of AACC over existing baselines experimentally in a range of simulated environments.
  arXiv  Detail & Related papers  (2022-08-03T22:52:26Z)
• Stronger Generalization Guarantees for Robot Learning by Combining Generative Models and Real-World Data [5.935761705025763]
  We provide a framework for providing generalization guarantees by leveraging a finite dataset of real-world environments. We demonstrate our approach on two simulated systems with nonlinear/hybrid dynamics and rich sensing modalities.
  arXiv  Detail & Related papers  (2021-11-16T20:13:10Z)
• Learning to Continuously Optimize Wireless Resource in a Dynamic Environment: A Bilevel Optimization Perspective [52.497514255040514]
  This work develops a new approach that enables data-driven methods to continuously learn and optimize resource allocation strategies in a dynamic environment. We propose to build the notion of continual learning into wireless system design, so that the learning model can incrementally adapt to the new episodes. Our design is based on a novel bilevel optimization formulation which ensures certain fairness" across different data samples.
  arXiv  Detail & Related papers  (2021-05-03T07:23:39Z)
• Towards Lifelong Learning of End-to-end ASR [81.15661413476221]
  Lifelong learning aims to enable a machine to sequentially learn new tasks from new datasets describing the changing real world without forgetting the previously learned knowledge. An overall relative reduction of 28.7% in WER was achieved compared to the fine-tuning baseline when sequentially learning on three very different benchmark corpora.
  arXiv  Detail & Related papers  (2021-04-04T13:48:53Z)
• Learning to Continuously Optimize Wireless Resource In Episodically Dynamic Environment [55.91291559442884]
  This work develops a methodology that enables data-driven methods to continuously learn and optimize in a dynamic environment. We propose to build the notion of continual learning into the modeling process of learning wireless systems. Our design is based on a novel min-max formulation which ensures certain fairness" across different data samples.
  arXiv  Detail & Related papers  (2020-11-16T08:24:34Z)
• Target-Embedding Autoencoders for Supervised Representation Learning [111.07204912245841]
  This paper analyzes a framework for improving generalization in a purely supervised setting, where the target space is high-dimensional. We motivate and formalize the general framework of target-embedding autoencoders (TEA) for supervised prediction, learning intermediate latent representations jointly optimized to be both predictable from features as well as predictive of targets.
  arXiv  Detail & Related papers  (2020-01-23T02:37:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.