Old Habits Die Hard: How Conversational History Geometrically Traps LLMs

• URL: http://arxiv.org/abs/2603.03308v1
• Date: Sun, 08 Feb 2026 14:13:15 GMT
• Title: Old Habits Die Hard: How Conversational History Geometrically Traps LLMs
• Abstract summary: Recent work suggests that large language models (LLMs) are affected by their conversational history in unexpected ways.<n>We introduce History-Echoes, a framework that investigates how conversational history biases subsequent generations.
• Score: 59.84224364012311
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: How does the conversational past of large language models (LLMs) influence their future performance? Recent work suggests that LLMs are affected by their conversational history in unexpected ways. For instance, hallucinations in prior interactions may influence subsequent model responses. In this work, we introduce History-Echoes, a framework that investigates how conversational history biases subsequent generations. The framework explores this bias from two perspectives: probabilistically, we model conversations as Markov chains to quantify state consistency; geometrically, we measure the consistency of consecutive hidden representations. Across three model families and six datasets spanning diverse phenomena, our analysis reveals a strong correlation between the two perspectives. By bridging these perspectives, we demonstrate that behavioral persistence manifests as a geometric trap, where gaps in the latent space confine the model's trajectory. Code available at https://github.com/technion-cs-nlp/OldHabitsDieHard.

Related papers

• MindTopo: Can Foundation Models Reason in Topological Space? [52.56750995613252]
  We introduce MindTopo, a benchmark of topological intuition across five properties grounded in cognitive science and formal topology.<n>MindTopo evaluates each property at two cognitive levels. Reasoning asks a model to identify topological relations or infer how they change.<n>Every MLLM performs better on reasoning than on planning, and the best-performing model remains far below observed human performance.
  arXiv  Detail & Related papers  (2026-09-10T17:54:32Z)
• POINTS-Seeker: Towards Training a Multimodal Agentic Search Model from Scratch [84.73366911912512]
  We introduce Agentic Seeding, a dedicated phase designed to weave the precursors necessary for eliciting agentic behaviors.<n>We propose V-Fold, an adaptive history-aware compression scheme that preserves recent dialogue turns in high fidelity while folding historical context into the visual space via rendering.<n>We develop POINTS-Seeker-8B, a state-of-the-art multimodal agentic search model that consistently outperforms existing models across six diverse benchmarks.
  arXiv  Detail & Related papers  (2026-04-15T16:09:37Z)
• Deep Autocorrelation Modeling for Time-Series Forecasting: Progress and Prospects [122.77185406890963]
  Autocorrelation is a defining characteristic of time-series data, where each observation is statistically dependent on its predecessors.<n>Recent studies have made strides in tackling these challenges, but a systematic survey examining both aspects remains lacking.<n>This paper provides a comprehensive review of deep time-series forecasting from the perspective of autocorrelation modeling.
  arXiv  Detail & Related papers  (2026-03-20T12:31:08Z)
• If an LLM Were a Character, Would It Know Its Own Story? Evaluating Lifelong Learning in LLMs [55.8331366739144]
  We introduce LIFESTATE-BENCH, a benchmark designed to assess lifelong learning in large language models (LLMs)<n>Our fact checking evaluation probes models' self-awareness, episodic memory retrieval, and relationship tracking, across both parametric and non-parametric approaches.
  arXiv  Detail & Related papers  (2025-03-30T16:50:57Z)
• Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspective [5.769786334333616]
  Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) based applications including automated text generation, question answering, and others. They face a significant challenge: hallucinations, where models produce plausible-sounding but factually incorrect responses. This paper discusses these open challenges covering state-of-the-art datasets and benchmarks as well as methods for knowledge integration and evaluating hallucinations.
  arXiv  Detail & Related papers  (2024-11-21T16:09:05Z)
• Integrating Large Language Models with Graph-based Reasoning for Conversational Question Answering [58.17090503446995]
  We focus on a conversational question answering task which combines the challenges of understanding questions in context and reasoning over evidence gathered from heterogeneous sources like text, knowledge graphs, tables, and infoboxes. Our method utilizes a graph structured representation to aggregate information about a question and its context.
  arXiv  Detail & Related papers  (2024-06-14T13:28:03Z)
• Imagination is All You Need! Curved Contrastive Learning for Abstract Sequence Modeling Utilized on Long Short-Term Dialogue Planning [2.5567566997688043]
  We introduce Curved Contrastive Learning (CCL), a novel representation learning technique for learning the relative turn distance between utterance pairs in multi-turn dialogues. The resulting bi-encoder models can guide transformers as a response ranking model towards a goal in a zero-shot fashion by projecting the goal utterance and the corresponding reply candidates into a latent space.
  arXiv  Detail & Related papers  (2022-11-14T18:16:48Z)
• Back to the Future: On Potential Histories in NLP [11.438985158220797]
  We argue that contemporary methods for machine learning are prejudiced towards dominant and hegemonic histories. By surfacing marginalized histories within contemporary conditions, we can create models that better represent the lived realities of traditionally marginalized and excluded communities.
  arXiv  Detail & Related papers  (2022-10-12T14:32:25Z)
• An Empirical Study: Extensive Deep Temporal Point Process [61.14164208094238]
  We first review recent research emphasis and difficulties in modeling asynchronous event sequences with deep temporal point process.<n>We propose a Granger causality discovery framework for exploiting the relations among multi-types of events.
  arXiv  Detail & Related papers  (2021-10-19T10:15:00Z)
• A Neural Conversation Generation Model via Equivalent Shared Memory Investigation [39.922967513749654]
  We propose a novel reading and memory framework called Deep Reading Memory Network (DRMN) DRMN is capable of remembering useful information of similar conversations for improving utterance generation. We apply our model to two large-scale conversation datasets of justice and e-commerce fields.
  arXiv  Detail & Related papers  (2021-08-20T13:20:14Z)
• Ranking Enhanced Dialogue Generation [77.8321855074999]
  How to effectively utilize the dialogue history is a crucial problem in multi-turn dialogue generation. Previous works usually employ various neural network architectures to model the history. This paper proposes a Ranking Enhanced Dialogue generation framework.
  arXiv  Detail & Related papers  (2020-08-13T01:49:56Z)
• Modality-Balanced Models for Visual Dialogue [102.35406085738325]
  The Visual Dialog task requires a model to exploit both image and conversational context information to generate the next response to the dialogue. We show that previous joint-modality (history and image) models over-rely on and are more prone to memorizing the dialogue history. We present methods for this integration of the two models, via ensemble and consensus dropout fusion with shared parameters.
  arXiv  Detail & Related papers  (2020-01-17T14:57:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.