Learning Personalized Memory Gating from Delayed One-Sided Feedback
Abstract Overview
This paper investigates personalized memory gating for LLM memory systems at the fact level, addressing which extracted facts should be stored under finite memory budgets. It introduces a retrieval-based labeling pipeline that identifies useful facts when they are subsequently retrieved and verified by an LLM judge as necessary or substantially helpful for future user turns. The authors formulate online memory gating as a contextual apple-tasting problem with one-sided delayed feedback, in which utility signals are observed only for stored facts and arrive after unknown delays. To solve this, they propose Warm-Start UCB, a two-stage method combining cross-user offline pretraining with online upper confidence bound adaptation for user-specific refinement.
Novelty
The work introduces a fact-level utility labeling pipeline grounded in downstream retrieval outcomes refined by an LLM judge, avoiding heuristic or downstream reward proxies. It also formulates personalized memory gating as an apple-tasting problem with one-sided delayed feedback and proposes Warm-Start UCB to balance cross-user prior knowledge with online adaptation.
Results
The authors establish a high-probability regret bound for Warm-Start UCB of order O~(min{Tε, (d√T + md)/κ_m}) under specified assumptions. Across evaluated memory budgets on PerMemBench-S and PerMemBench-L, Warm-Start UCB consistently outperforms universal admit-all storage and LLM-based gating baselines. On PerMemBench-L, it achieves the best non-oracle retrieval-hit retention in eight of ten evaluation settings, with ablations confirming cumulative gains from both offline pretraining pool size and online feedback adaptation.
Key Points
- A retrieval-based labeling pipeline generates ground-truth utility labels for memory facts using downstream retrieval hits and LLM-judge verification.
- Personalized fact gating is modeled as a contextual apple-tasting problem with one-sided delayed feedback and addressed using the two-stage Warm-Start UCB algorithm.
- Empirical evaluations on PerMemBench-S and PerMemBench-L demonstrate superior retrieval-hit retention over universal admit-all, frozen pretrained scorers, cold-start UCB, and LLM-based baselines.