Future of AI Models: A Computational perspective on Model collapse

• URL: http://arxiv.org/abs/2511.05535v1
• Date: Wed, 29 Oct 2025 15:10:29 GMT
• Title: Future of AI Models: A Computational perspective on Model collapse
• Abstract summary: This study quantifies and forecasts collapse onset by examining year-wise semantic similarity in English-language Wikipedia from 2013 to 2025.<n>Results reveal a steady rise in similarity before public LLM adoption, likely driven by early RNN/LSTM and text-normalization pipelines.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial Intelligence, especially Large Language Models (LLMs), has transformed domains such as software engineering, journalism, creative writing, academia, and media (Naveed et al. 2025; arXiv:2307.06435). Diffusion models like Stable Diffusion generate high-quality images and videos from text. Evidence shows rapid expansion: 74.2% of newly published webpages now contain AI-generated material (Ryan Law 2025), 30-40% of the active web corpus is synthetic (Spennemann 2025; arXiv:2504.08755), 52% of U.S. adults use LLMs for writing, coding, or research (Staff 2025), and audits find AI involvement in 18% of financial complaints and 24% of press releases (Liang et al. 2025). The underlying neural architectures, including Transformers (Vaswani et al. 2023; arXiv:1706.03762), RNNs, LSTMs, GANs, and diffusion networks, depend on large, diverse, human-authored datasets (Shi & Iyengar 2019). As synthetic content dominates, recursive training risks eroding linguistic and semantic diversity, producing Model Collapse (Shumailov et al. 2024; arXiv:2307.15043; Dohmatob et al. 2024; arXiv:2402.07712). This study quantifies and forecasts collapse onset by examining year-wise semantic similarity in English-language Wikipedia (filtered Common Crawl) from 2013 to 2025 using Transformer embeddings and cosine similarity metrics. Results reveal a steady rise in similarity before public LLM adoption, likely driven by early RNN/LSTM translation and text-normalization pipelines, though modest due to a smaller scale. Observed fluctuations reflect irreducible linguistic diversity, variable corpus size across years, finite sampling error, and an exponential rise in similarity after the public adoption of LLM models. These findings provide a data-driven estimate of when recursive AI contamination may significantly threaten data richness and model generalization.

Related papers

• Beyond "AI Language": The case for the idiolectal nature of LLM output [0.0]
  Large language model outputs are frequently analysed as a collective super variety termed "AI language"<n>This chapter argues that this perspective coexists with distinct, model-specific linguistic signatures akin to human idiolects.
  arXiv  Detail & Related papers  (2026-08-06T21:03:13Z)
• Speaking the Language of Science: Toward a General-Purpose Generative Foundation Model for the Natural Sciences [89.71422932447423]
  LOGOS is a scientific generative language model that unifies heterogeneous tasks across the natural sciences.<n>By representing spatial contact and constraint patterns as discrete tokens, the model captures complex structural interactions in a purely sequential manner.<n>Across diverse tasks, LOGOS consistently matches or outperforms domain-specific baselines.
  arXiv  Detail & Related papers  (2026-06-15T16:14:53Z)
• Topical Phase Transitions in Artificial Intelligence Research: Large-Scale Evidence and an Early-Warning Signature for Emerging Topics [0.8594140167290097]
  We analyze 80,814 accepted main-track papers from five premier AI conferences spanning 2017 to 2025.<n>We show major AI topics advance through topical phase transitions: remaining marginal for years, then surging across venues within one to three years.<n>This structure is our primary contribution: a large-scale, cross-venue characterization of how AI research reorganizes.
  arXiv  Detail & Related papers  (2026-06-11T02:47:41Z)
• Filling in the Mechanisms: How do LMs Learn Filler-Gap Dependencies under Developmental Constraints? [0.31511847280063693]
  We evaluate whether representations of filler-gap dependencies transfer between wh-questions and topicalization.<n>Our results suggest shared, yet item-sensitive mechanisms may develop with limited training data.<n>More importantly, LMs still require far more data than humans to learn comparable generalizations.
  arXiv  Detail & Related papers  (2026-04-15T22:32:22Z)
• The Cognitive Divergence: AI Context Windows, Human Attention Decline, and the Delegation Feedback Loop [0.0]
  AI context windows have grown from 512 tokens in 2017 to 2 tokens by 2026.<n>Human Effective Context Span has declined from 16,000 tokens to 1,800.<n>The AI-to-human ratio grew from near parity at the ChatGPT launch to 556--1,111x raw and 56-111x quality-adjusted, after accounting for retrieval degradation.
  arXiv  Detail & Related papers  (2026-03-17T18:53:45Z)
• From Memorization to Creativity: LLM as a Designer of Novel Neural-Architectures [48.83701310501069]
  Large language models (LLMs) excel in program synthesis, yet their ability to autonomously navigate neural architecture design-balancing reliability, performance, and structural novelty--remains underexplored.<n>We address this by placing a code-oriented LLM within a closed-loop synthesis framework, analyzing its evolution over 22 supervised fine-tuning cycles.
  arXiv  Detail & Related papers  (2026-01-06T13:20:28Z)
• Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond) [90.45301024940329]
  Language models (LMs) often struggle to generate diverse, human-like creative content.<n>We introduce Infinity-Chat, a large-scale dataset of 26K diverse, real-world, open-ended user queries.<n>We present a large-scale study of mode collapse in LMs, revealing a pronounced Artificial Hivemind effect.
  arXiv  Detail & Related papers  (2025-10-27T03:16:21Z)
• Does GenAI Rewrite How We Write? An Empirical Study on Two-Million Preprints [15.070885964897734]
  Generative large language models (LLMs) introduce a further potential disruption by altering how manuscripts are written.<n>This paper addresses the gap through a large-scale analysis of more than 2.1 million preprints spanning 2016--2025 (115 months) across four major repositories.<n>Our findings reveal that LLMs have accelerated submission and revision cycles, modestly increased linguistic complexity, and disproportionately expanded AI-related topics.
  arXiv  Detail & Related papers  (2025-10-18T01:37:40Z)
• Riemannian-Geometric Fingerprints of Generative Models [10.098284109691138]
  We propose a new definition of artifact and fingerprint of generative models (GMs)<n>We apply our theory to a new gradient-based algorithm for computing the fingerprints in practice.<n>Results show that it is more effective in distinguishing a large array of GMs, spanning across 4 different datasets in 2 different resolutions.
  arXiv  Detail & Related papers  (2025-06-28T08:08:16Z)
• Vectors from Larger Language Models Predict Human Reading Time and fMRI Data More Poorly when Dimensionality Expansion is Controlled [8.414116316164888]
  The impressive linguistic abilities of large language models (LLMs) have recommended them as models of human sentence processing.<n>Recent studies have shown this scaling inverts after a point, as LMs become excessively large and accurate, when word prediction probability is used as a predictor.<n>This study evaluates LLM scaling using entire LLM vectors, while controlling for the larger number of predictors in vectors from larger LLMs.
  arXiv  Detail & Related papers  (2025-05-18T02:13:48Z)
• Could AI Trace and Explain the Origins of AI-Generated Images and Text? [53.11173194293537]
  AI-generated content is increasingly prevalent in the real world.<n> adversaries might exploit large multimodal models to create images that violate ethical or legal standards.<n>Paper reviewers may misuse large language models to generate reviews without genuine intellectual effort.
  arXiv  Detail & Related papers  (2025-04-05T20:51:54Z)
• What fifty-one years of Linguistics and Artificial Intelligence research tell us about their correlation: A scientometric analysis [0.0]
  This study provides a thorough scientometric analysis of this correlation, synthesizing the intellectual production over 51 years, from 1974 to 2024.<n>The results indicate that in the 1980s and 1990s, linguistics and AI (AIL) research was not robust, characterized by unstable publication over time.<n>It concludes that linguistics and AI correlation is established at several levels, research centers, journals, and countries shaping AIL knowledge production and reshaping its future frontiers.
  arXiv  Detail & Related papers  (2024-11-29T17:12:06Z)
• In-Context Language Learning: Architectures and Algorithms [73.93205821154605]
  We study ICL through the lens of a new family of model problems we term in context language learning (ICLL) We evaluate a diverse set of neural sequence models on regular ICLL tasks.
  arXiv  Detail & Related papers  (2024-01-23T18:59:21Z)
• Scaling Laws Do Not Scale [54.72120385955072]
  Recent work has argued that as the size of a dataset increases, the performance of a model trained on that dataset will increase. We argue that this scaling law relationship depends on metrics used to measure performance that may not correspond with how different groups of people perceive the quality of models' output. Different communities may also have values in tension with each other, leading to difficult, potentially irreconcilable choices about metrics used for model evaluations.
  arXiv  Detail & Related papers  (2023-07-05T15:32:21Z)
• The Curse of Recursion: Training on Generated Data Makes Models Forget [70.02793975243212]
  Large language models (LLMs) are here to stay, and will bring about drastic change in the whole ecosystem of online text and images. We find that use of model-generated content in training causes irreversible defects in the resulting models, where tails of the original content distribution disappear.
  arXiv  Detail & Related papers  (2023-05-27T15:10:41Z)
• A comprehensive comparative evaluation and analysis of Distributional Semantic Models [61.41800660636555]
  We perform a comprehensive evaluation of type distributional vectors, either produced by static DSMs or obtained by averaging the contextualized vectors generated by BERT. The results show that the alleged superiority of predict based models is more apparent than real, and surely not ubiquitous. We borrow from cognitive neuroscience the methodology of Representational Similarity Analysis (RSA) to inspect the semantic spaces generated by distributional models.
  arXiv  Detail & Related papers  (2021-05-20T15:18:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.