EvoEmbedding: Evolvable Representations for Long-Context Retrieval and Agentic Memory

• URL: http://arxiv.org/abs/2606.21649v2
• Date: Thu, 25 Jun 2026 08:16:13 GMT
• Title: EvoEmbedding: Evolvable Representations for Long-Context Retrieval and Agentic Memory
• Abstract summary: EvoEmbedding is a novel embedding model that generates evolvable representations for retrieval.<n>It is tailored for long-context scenarios, where information is dynamic, sequential, and requires continuous state tracking.<n>EvoEmbedding seamlessly integrates into agentic to boost performance.
• Score: 46.411965241105804
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing embedding models are inherently static: they encode text segments in isolation, ignoring their surrounding context and temporal order. This paper introduces EvoEmbedding, a novel embedding model that generates evolvable representations for retrieval. It is tailored for long-context scenarios, where information is dynamic, sequential, and requires continuous state tracking. Our design is simple: EvoEmbedding maintains a continuously updated latent memory as it sequentially processes inputs, and uses it alongside the raw content to jointly generate evolvable embeddings. Consequently, for the same query, our model adapts its representation to retrieve distinct targets based on the evolving context, going beyond static semantic search. To equip the model with this capability, we construct EvoTrain-180K, a diverse dataset for the joint optimization of latent memory and retrieval. Furthermore, we introduce a memory queue to prevent representation collapse during recurrent encoding, alongside segment-batching techniques that tackle significant length variance and accelerate training by 3.8$\times$. Extensive experiments show that our model not only outperforms larger-scale specialists (e.g., Qwen3-Embedding-8B and KaLM-Embedding-Gemma3-12B) across a range of long-context retrieval benchmarks, but also generalizes well to downstream tasks (e.g., personalization) with contexts 10$\times$ longer than its training window. Notably, EvoEmbedding seamlessly integrates into agentic workflows to boost performance. For instance, a naive RAG pipeline equipped with our model surpasses dedicated agentic memory systems. Project Page: https://clare-nie.github.io/EvoEmbedding/.

Related papers

• ConvMem: Convolutional Memory for Long-Context Reasoning [18.597196274231553]
  ConvMem is a training-free, highly parallelizable framework that reformulates long-context reasoning as a hierarchical convolution.<n>We show that ConvMem outperforms training-free baselines and avoids the risk of overfitting to parametric priors on out-of-distribution tasks.
  arXiv  Detail & Related papers  (2026-09-09T16:53:02Z)
• From Atom to Community: Structured and Evolving Agent Memory for User Behavior Modeling [25.74573956240653]
  STEAM (textittextbfSTructured and textbfEvolving textbfAgent textbfMemory) is a novel framework that reimagines how agent memory is organized and updated.<n>It decomposes preferences into atomic memory units, each capturing a distinct interest dimension with explicit links to observed behaviors.<n>To exploit collaborative patterns, STEAM organizes similar memories across users into communities and generates prototype memories for signal propagation.
  arXiv  Detail & Related papers  (2026-01-23T16:24:57Z)
• Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory [89.65731902036669]
  Evo-Memory is a streaming benchmark and framework for evaluating self-evolving memory in large language model (LLM) agents.<n>We evaluate over ten representative memory modules and evaluate them across 10 diverse multi-turn goal-oriented and single-turn reasoning and QA datasets.
  arXiv  Detail & Related papers  (2025-11-25T21:08:07Z)
• SitEmb-v1.5: Improved Context-Aware Dense Retrieval for Semantic Association and Long Story Comprehension [77.93156509994994]
  We show how to represent short chunks in a way that is conditioned on a broader context window to enhance retrieval performance.<n>Existing embedding models are not well-equipped to encode such situated context effectively.<n>Our method substantially outperforms state-of-the-art embedding models.
  arXiv  Detail & Related papers  (2025-08-03T23:59:31Z)
• Leveraging Machine Learning and Enhanced Parallelism Detection for BPMN Model Generation from Text [75.77648333476776]
  This paper introduces an automated pipeline for extracting BPMN models from text.<n>A key contribution of this work is the introduction of a newly annotated dataset.<n>We augment the dataset with 15 newly annotated documents containing 32 parallel gateways for model training.
  arXiv  Detail & Related papers  (2025-07-11T07:25:55Z)
• PersonalAI: A Systematic Comparison of Knowledge Graph Storage and Retrieval Approaches for Personalized LLM agents [15.524189150821147]
  Large language models (LLMs) combined with Retrieval-Augmented Generation (RAG) fail to scale in complex, long-term interactions.<n>We propose a flexible external memory framework based on knowledge graphs, automatically constructed and updated by the LLM itself.<n>Building upon the AriGraph architecture, we introduce a novel hybrid graph design that supports both standard edges and two types of hyperedges.<n>We evaluate our system on three benchmarks-TriviaQA, HotpotQA, and DiaASQ-demonstrating that different memory and retrieval configurations yield optimal performance depending on the task.
  arXiv  Detail & Related papers  (2025-06-20T13:52:15Z)
• KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark of Long Context Capable Approaches [52.02764371205856]
  Long context capability is a crucial competency for large language models (LLMs) This work provides a taxonomy of current methods and evaluating 10+ state-of-the-art approaches across seven categories of long context tasks.
  arXiv  Detail & Related papers  (2024-07-01T17:59:47Z)
• Autoregressive Search Engines: Generating Substrings as Document Identifiers [53.0729058170278]
  Autoregressive language models are emerging as the de-facto standard for generating answers. Previous work has explored ways to partition the search space into hierarchical structures. In this work we propose an alternative that doesn't force any structure in the search space: using all ngrams in a passage as its possible identifiers.
  arXiv  Detail & Related papers  (2022-04-22T10:45:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.