Does Episodic Memory Help Close the Lexical Frequency Gap in Sensitivity to Syntactic Contrasts? A Test Using Retrieval-Augmented Language Models

• URL: http://arxiv.org/abs/2608.23851v1
• Date: Mon, 24 Aug 2026 21:45:12 GMT
• Title: Does Episodic Memory Help Close the Lexical Frequency Gap in Sensitivity to Syntactic Contrasts? A Test Using Retrieval-Augmented Language Models
• Abstract summary: We use Complementary Learning Systems theory to test the hypothesis that to lexical frequency can arise via a hippocampal episodic memory mechanism.<n>We find that retrieval augmentation narrows the performance gap between high- and low-frequency items.<n>We show that structural information is critical for effective retrieval, whereas semantic similarity alone provides little benefit.
• Score: 13.255966254489842
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Grammatical knowledge and how it is empirically tested are typically considered robust to the frequency of the lexical items in the expressions. However, neural network-based models of grammaticality exhibit high sensitivity to lexical frequency. We draw upon Complementary Learning Systems theory to test the hypothesis that robustness to lexical frequency can arise via a hippocampal episodic memory mechanism, which enables rapid encoding and retrieval of specific experiences and allows learners to leverage them when processing rare patterns. We use retrieval-augmented language models as an instantiation of such an episodic memory mechanism (specifically, $k$-nearest-neighbor language models that augment parametric models with explicit instance storage), and test whether this augmentation helps close the lexical frequency gap that vanilla language models exhibit in syntactic contrast tests. Using syntactic contrasts with frequency-stratified test items, we find that retrieval augmentation narrows the performance gap between high- and low-frequency items, consistent with episodic memory compensating for weak parametric representations. This benefit is consistent across different syntactic phenomena and across models pretrained on child-realistic and large-scale data. Additionally, we show that structural information is critical for effective retrieval, whereas semantic similarity alone provides little benefit. While these are promising proof-of-concept results supporting our hypothesis, the frequency gap is narrowed rather than fully closed. Based on our analyses, we propose preferential reweighting of retrieved instances, better representations and retrieval strategies for structural information, and flexible configurations of storage and retrieval as promising future directions for improving the implementation of episodic memory in language models.

Related papers

• Characterizing Memorization in Diffusion Language Models: Generalized Extraction and Sampling Effects [17.220195638215507]
  Diffusion language models (DLMs) have emerged as a competitive alternative to autoregressive language models (ARMs)<n>DLMs exhibit substantially lower memorization-based leakage of personally identifiable information (PII) compared to ARMs.
  arXiv  Detail & Related papers  (2026-03-02T19:03:32Z)
• Generative vector search to improve pathology foundation models across multimodal vision-language tasks [0.0]
  We present Latent Matching (STHLM), a generative vector search method that samples query-conditioned embeddings to enhance retrieval performance.<n>STHLM demonstrates critical improvements over classical vector retrieval across diverse benchmarks, including scientific literature, clinical notes, and tissue images.
  arXiv  Detail & Related papers  (2025-12-22T12:59:23Z)
• From Fuzzy Speech to Medical Insight: Benchmarking LLMs on Noisy Patient Narratives [40.12543056558646]
  We present a novel dataset designed to simulate patient self-descriptions characterized by varying levels of linguistic noise, fuzzy language, and layperson terminology.<n>Our dataset comprises clinically consistent scenarios annotated with ground-truth diagnoses, spanning a spectrum of communication clarity to reflect diverse real-world reporting styles.<n>To support and future research, we release the Noisy Diagnostic Benchmark (NDB), a structured dataset of noisy, synthetic patient descriptions designed to stress-test and compare the diagnostic capabilities of large language models (LLMs) under realistic linguistic conditions.
  arXiv  Detail & Related papers  (2025-09-15T11:34:46Z)
• SynapticRAG: Enhancing Temporal Memory Retrieval in Large Language Models through Synaptic Mechanisms [8.787174594966492]
  We propose SynapticRAG, which combines temporal association triggers with biologically-inspired synaptic propagation mechanisms.<n>Our approach uses temporal association triggers and synaptic-like stimulus propagation to identify relevant dialogue histories.<n>Experiments on four datasets show that SynapticRAG achieves consistent improvements across multiple metrics up to 14.66% points.
  arXiv  Detail & Related papers  (2024-10-17T13:51:03Z)
• Contrastive Learning with Counterfactual Explanations for Radiology Report Generation [83.30609465252441]
  We propose a textbfCountertextbfFactual textbfExplanations-based framework (CoFE) for radiology report generation. Counterfactual explanations serve as a potent tool for understanding how decisions made by algorithms can be changed by asking what if'' scenarios. Experiments on two benchmarks demonstrate that leveraging the counterfactual explanations enables CoFE to generate semantically coherent and factually complete reports.
  arXiv  Detail & Related papers  (2024-07-19T17:24:25Z)
• Improving the Robustness of Summarization Systems with Dual Augmentation [68.53139002203118]
  A robust summarization system should be able to capture the gist of the document, regardless of the specific word choices or noise in the input. We first explore the summarization models' robustness against perturbations including word-level synonym substitution and noise. We propose a SummAttacker, which is an efficient approach to generating adversarial samples based on language models.
  arXiv  Detail & Related papers  (2023-06-01T19:04:17Z)
• Scaling laws for language encoding models in fMRI [47.498241053872924]
  We tested whether larger open-source models are better at predicting brain responses recorded using fMRI. Similar logarithmic behavior was observed when scaling the size of the fMRI training set. These results suggest that increasing scale in both models and data will yield incredibly effective models of language processing in the brain.
  arXiv  Detail & Related papers  (2023-05-19T17:53:03Z)
• An Empirical Investigation of Commonsense Self-Supervision with Knowledge Graphs [67.23285413610243]
  Self-supervision based on the information extracted from large knowledge graphs has been shown to improve the generalization of language models. We study the effect of knowledge sampling strategies and sizes that can be used to generate synthetic data for adapting language models.
  arXiv  Detail & Related papers  (2022-05-21T19:49:04Z)
• On The Ingredients of an Effective Zero-shot Semantic Parser [95.01623036661468]
  We analyze zero-shot learning by paraphrasing training examples of canonical utterances and programs from a grammar. We propose bridging these gaps using improved grammars, stronger paraphrasers, and efficient learning methods. Our model achieves strong performance on two semantic parsing benchmarks (Scholar, Geo) with zero labeled data.
  arXiv  Detail & Related papers  (2021-10-15T21:41:16Z)
• Evaluating Models of Robust Word Recognition with Serial Reproduction [8.17947290421835]
  We compare several broad-coverage probabilistic generative language models in their ability to capture human linguistic expectations. We find that those models that make use of abstract representations of preceding linguistic context best predict the changes made by people in the course of serial reproduction.
  arXiv  Detail & Related papers  (2021-01-24T20:16:12Z)
• Multi-timescale Representation Learning in LSTM Language Models [69.98840820213937]
  Language models must capture statistical dependencies between words at timescales ranging from very short to very long. We derived a theory for how the memory gating mechanism in long short-term memory language models can capture power law decay. Experiments showed that LSTM language models trained on natural English text learn to approximate this theoretical distribution.
  arXiv  Detail & Related papers  (2020-09-27T02:13:38Z)
• Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans [75.15855405318855]
  We studied whether a modern artificial neural network trained with "deep learning" methods mimics a central aspect of human sentence processing. Although the network was solely trained to predict the next word in a large corpus, analysis showed the emergence of specialized units that successfully handled local and long-distance syntactic agreement. We tested the model's predictions in a behavioral experiment where humans detected violations in number agreement in sentences with systematic variations in the singular/plural status of multiple nouns.
  arXiv  Detail & Related papers  (2020-06-19T12:00:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.