The Narrative Continuity Test: A Conceptual Framework for Evaluating Identity Persistence in AI Systems

• URL: http://arxiv.org/abs/2510.24831v2
• Date: Sat, 01 Nov 2025 10:06:59 GMT
• Title: The Narrative Continuity Test: A Conceptual Framework for Evaluating Identity Persistence in AI Systems
• Abstract summary: Narrative Continuity Test (NCT) is a framework for evaluating identity persistence and diachronic coherence in AI systems.<n>NCT defines five necessary axes -- Situated Memory, Goal Persistence, Autonomous Self-Correction, Stylistic & Semantic Stability, and Persona/Role Continuity.<n>Case analyses show predictable continuity failures under stateless inference.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial intelligence systems based on large language models (LLMs) can now generate coherent text, music, and images, yet they operate without a persistent state: each inference reconstructs context from scratch. This paper introduces the Narrative Continuity Test (NCT) -- a conceptual framework for evaluating identity persistence and diachronic coherence in AI systems. Unlike capability benchmarks that assess task performance, the NCT examines whether an LLM remains the same interlocutor across time and interaction gaps. The framework defines five necessary axes -- Situated Memory, Goal Persistence, Autonomous Self-Correction, Stylistic & Semantic Stability, and Persona/Role Continuity -- and explains why current architectures systematically fail to support them. Case analyses (Character.\,AI, Grok, Replit, Air Canada) show predictable continuity failures under stateless inference. The NCT reframes AI evaluation from performance to persistence, outlining conceptual requirements for future benchmarks and architectural designs that could sustain long-term identity and goal coherence in generative models.

Related papers

• Multi-Dimensional Assessment for AI Cognition (MAAC): A Theoretical Framework for Process-Oriented Cognitive Evaluation of Text-Based AI Systems [1.8262547855491456]
  This paper introduces the Multi-Dimensional Assessment for AI Cognition (MAAC), a framework for shifting evaluation from what text-based AI systems produce to how they think.<n>MAAC defines nine cognitively motivated dimensions: Cognitive Load, Tool Execution, Content Quality, Memory Integration, Complexity Handling, Hallucination Control, Knowledge Transfer, Processing Efficiency, and Process-Outcome Alignment.<n>Five theoretical analyses provide initial support for the framework's coherence and empirical testability.
  arXiv  Detail & Related papers  (2026-08-01T14:01:17Z)
• From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape [79.30826980815927]
  rubrics are explicit criteria sets that transform complex quality judgments into structured and actionable standards.<n>We systematically organize existing rubric designs, examine their construction and optimization, and analyze their role across evaluation and training.
  arXiv  Detail & Related papers  (2026-06-07T13:34:55Z)
• A Persona-Based Evaluation Framework for Pluralistic Alignment in Generative AI [0.0]
  We introduce a state-space constrained emulation framework for AI evaluation that replaces singular assessment functions with a structured manifold of synthetic cognitive profiles representing diverse human perspectives.<n>We show that modern generative architectures can instantiate and maintain these evaluative personas with high consistency.<n>This study provides a foundation for more adaptive, human-aligned, and context-sensitive approaches to AI evaluation.
  arXiv  Detail & Related papers  (2026-05-29T08:54:09Z)
• Beyond Accuracy: Benchmarking Cross-Task Consistency in Unified Multimodal Models [12.660879089125684]
  Unified Multimodal Models (uMMs) aim to support both visual understanding and visual generation within a shared representation.<n>Existing evaluation protocols assess these two capabilities independently and do not examine whether they are semantically aligned.<n>We introduce XTC-Bench, a scene-graph-grounded evaluation framework that measures cross-task visual semantic consistency.
  arXiv  Detail & Related papers  (2026-04-27T23:57:29Z)
• SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models [86.19617358080016]
  Social Omni is a benchmark that operationalizes the evaluation of conversational interactivity across three core dimensions.<n>Social Omni features 2,000 perception samples and a quality-controlled diagnostic set of 209 interaction-generation instances.<n>Our analysis reveals a pronounced decoupling between a model's perceptual accuracy and its ability to generate contextually appropriate interruptions.
  arXiv  Detail & Related papers  (2026-03-17T17:58:44Z)
• The Path Ahead for Agentic AI: Challenges and Opportunities [4.52683540940001]
  This chapter examines the emergence of agentic AI systems that operate autonomously in complex environments.<n>We trace the architectural progression from statistical models to transformer-based systems, identifying capabilities that enable agentic behavior.<n>Unlike existing surveys, we focus on the architectural transition from language understanding to autonomous action, emphasizing the technical gaps that must be resolved before deployment.
  arXiv  Detail & Related papers  (2026-01-06T06:31:42Z)
• Can World Simulators Reason? Gen-ViRe: A Generative Visual Reasoning Benchmark [48.02995109011304]
  Video generation models have emerged as potential world simulators through Chain-of-Frames (CoF) reasoning.<n>Existing benchmarks, focusing on fidelity or alignment, do not assess CoF reasoning.<n>We introduce Gen-ViRe, a framework grounded in cognitive science and real-world AI applications.
  arXiv  Detail & Related papers  (2025-11-17T19:11:39Z)
• Priors in Time: Missing Inductive Biases for Language Model Interpretability [58.07412640266836]
  We show that Sparse Autoencoders impose priors that assume independence of concepts across time, implying stationarity.<n>We introduce a new interpretability objective -- Temporal Feature Analysis -- which possesses a temporal inductive bias to decompose representations at a given time into two parts.<n>Our results underscore the need for inductive biases that match the data in designing robust interpretability tools.
  arXiv  Detail & Related papers  (2025-11-03T18:43:48Z)
• PISA: A Pragmatic Psych-Inspired Unified Memory System for Enhanced AI Agency [50.712873697511206]
  Existing work often lacks adaptability to diverse tasks and overlooks the constructive and task-oriented role of AI agent memory.<n>We propose PISA, a pragmatic, psych-inspired unified memory system that treats memory as a constructive and adaptive process.<n>Our empirical evaluation, conducted on the existing LOCOMO benchmark and our newly proposed AggQA benchmark for data analysis tasks, confirms that PISA sets a new state-of-the-art by significantly enhancing adaptability and long-term knowledge retention.
  arXiv  Detail & Related papers  (2025-10-12T10:34:35Z)
• Analyzing Latent Concepts in Code Language Models [10.214183897113118]
  We propose Code Concept Analysis (CoCoA): a global post-hoc interpretability framework.<n>CoCoA uncovers emergent lexical, syntactic, and semantic structures in a code language model's representation space.<n>We propose a hybrid annotation pipeline that combines static analysis tool-based syntactic alignment with prompt-engineered large language models.
  arXiv  Detail & Related papers  (2025-10-01T03:53:21Z)
• Game-Time: Evaluating Temporal Dynamics in Spoken Language Models [93.844257719952]
  We introduce the Game-Time Benchmark framework to assess temporal capabilities.<n>Our evaluation of diverse SLM models reveals a clear performance disparity.<n>The GameTime Benchmark provides a foundation for guiding future research toward more temporally-aware conversational AI.
  arXiv  Detail & Related papers  (2025-09-30T15:23:39Z)
• CRACQ: A Multi-Dimensional Approach To Automated Document Assessment [0.0]
  CRACQ is a multi-dimensional evaluation framework tailored to evaluate documents across f i v e specific traits: Coherence, Rigor, Appropriateness, Completeness, and Quality.<n>It integrates linguistic, semantic, and structural signals into a cumulative assessment, enabling both holistic and trait-level analysis.
  arXiv  Detail & Related papers  (2025-09-26T17:01:54Z)
• Modeling and Visualization Reasoning for Stakeholders in Education and Industry Integration Systems: Research on Structured Synthetic Dialogue Data Generation Based on NIST Standards [3.5516803380598074]
  This study addresses the structural complexity and semantic ambiguity in stakeholder interactions within the Education-Industry Integration (EII) system.<n>We propose a structural modeling paradigm based on the National Institute of Standards and Technology (NIST) synthetic data quality framework.
  arXiv  Detail & Related papers  (2025-06-20T12:37:43Z)
• EOC-Bench: Can MLLMs Identify, Recall, and Forecast Objects in an Egocentric World? [52.99661576320663]
  multimodal large language models (MLLMs) have driven breakthroughs in egocentric vision applications.<n>EOC-Bench is an innovative benchmark designed to systematically evaluate object-centric embodied cognition in dynamic egocentric scenarios.<n>We conduct comprehensive evaluations of various proprietary, open-source, and object-level MLLMs based on EOC-Bench.
  arXiv  Detail & Related papers  (2025-06-05T17:44:12Z)
• A Controllable Examination for Long-Context Language Models [62.845852724511964]
  This study introduces $textbfLongBioBench, a benchmark for evaluating long-context language models.<n>We show that most models still exhibit deficiencies in semantic understanding and elementary reasoning over retrieved results.<n>Our further analysis indicates some design choices employed by existing synthetic benchmarks, such as contextual non-coherence.
  arXiv  Detail & Related papers  (2025-06-03T14:23:06Z)
• WHEN TO ACT, WHEN TO WAIT: Modeling the Intent-Action Alignment Problem in Dialogue [23.39224016484548]
  Dialogue systems often fail when user utterances are semantically complete yet lack clarity and completeness required for appropriate system action.<n>We present STORM, a framework modeling asymmetric information dynamics through conversations between UserLLM and AgentLLM.<n>Our contributions include: (1) formalizing asymmetric information processing in dialogue systems; (2) modeling intent formation tracking collaborative understanding evolution; and (3) evaluation metrics measuring internal cognitive improvements alongside task performance.
  arXiv  Detail & Related papers  (2025-06-02T17:11:10Z)
• Detecting Neurocognitive Disorders through Analyses of Topic Evolution and Cross-modal Consistency in Visual-Stimulated Narratives [83.15653194899126]
  Early detection of neurocognitive disorders (NCDs) is crucial for timely intervention and disease management.<n>Current VSN-based NCD detection methods primarily focus on linguistic microstructures closely tied to bottom-up, stimulus-driven cognitive processes.<n>We propose two novel macrostructural approaches: a Dynamic Topic Model (DTM) to track topic evolution over time, and a Text-Image Temporal Alignment Network (TITAN) to measure cross-modal consistency between narrative and visual stimuli.
  arXiv  Detail & Related papers  (2025-01-07T12:16:26Z)
• Benchmarks as Microscopes: A Call for Model Metrology [76.64402390208576]
  Modern language models (LMs) pose a new challenge in capability assessment. To be confident in our metrics, we need a new discipline of model metrology.
  arXiv  Detail & Related papers  (2024-07-22T17:52:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.