Characterizing Model Collapse in Large Language Models Using Semantic Networks and Next-Token Probability

• URL: http://arxiv.org/abs/2410.12341v2
• Date: Sun, 02 Feb 2025 22:40:09 GMT
• Title: Characterizing Model Collapse in Large Language Models Using Semantic Networks and Next-Token Probability
• Authors: Daniele Gambetta, Gizem Gezici, Fosca Giannotti, Dino Pedreschi, Alistair Knott, Luca Pappalardo,
• Abstract summary: As synthetic content increasingly infiltrates the web, generative AI models may experience an autophagy process, where they are fine-tuned using their own outputs.<n>This could lead to a phenomenon known as model collapse, which entails a degradation in the performance and diversity of generative AI models over successive generations.<n>Recent studies have explored the emergence of model collapse across various generative AI models and types of data.
• Score: 4.841442157674423
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As synthetic content increasingly infiltrates the web, generative AI models may experience an autophagy process, where they are fine-tuned using their own outputs. This autophagy could lead to a phenomenon known as model collapse, which entails a degradation in the performance and diversity of generative AI models over successive generations. Recent studies have explored the emergence of model collapse across various generative AI models and types of data. However, the current characterizations of model collapse tend to be simplistic and lack comprehensive evaluation. In this article, we conduct a thorough investigation of model collapse across three text datasets, utilizing semantic networks to analyze text repetitiveness and diversity, while employing next-token probabilities to quantify the loss of diversity. We also examine how the proportions of synthetic tokens affect the severity of model collapse and perform cross-dataset evaluations to identify domain-specific variations. By proposing metrics and strategies for a more detailed assessment of model collapse, our study provides new insights for the development of robust generative AI systems.

Related papers

• Harnessing the Intrinsic Knowledge of Pretrained Language Models for Challenging Text Classification Settings [5.257719744958367]
  This thesis explores three challenging settings in text classification by leveraging the intrinsic knowledge of pretrained language models (PLMs) We develop models that utilize features based on contextualized word representations from PLMs, achieving performance that rivals or surpasses human accuracy. Lastly, we tackle the sensitivity of large language models to in-context learning prompts by selecting effective demonstrations.
  arXiv  Detail & Related papers  (2024-08-28T09:07:30Z)
• Creativity Has Left the Chat: The Price of Debiasing Language Models [1.223779595809275]
  We investigate the unintended consequences of Reinforcement Learning from Human Feedback on the creativity of Large Language Models (LLMs) Our findings have significant implications for marketers who rely on LLMs for creative tasks such as copywriting, ad creation, and customer persona generation.
  arXiv  Detail & Related papers  (2024-06-08T22:14:51Z)
• Corpus Considerations for Annotator Modeling and Scaling [9.263562546969695]
  We show that the commonly used user token model consistently outperforms more complex models. Our findings shed light on the relationship between corpus statistics and annotator modeling performance.
  arXiv  Detail & Related papers  (2024-04-02T22:27:24Z)
• generAItor: Tree-in-the-Loop Text Generation for Language Model Explainability and Adaptation [28.715001906405362]
  Large language models (LLMs) are widely deployed in various downstream tasks, e.g., auto-completion, aided writing, or chat-based text generation. We tackle this shortcoming by proposing a tree-in-the-loop approach, where a visual representation of the beam search tree is the central component for analyzing, explaining, and adapting the generated outputs. We present generAItor, a visual analytics technique, augmenting the central beam search tree with various task-specific widgets, providing targeted visualizations and interaction possibilities.
  arXiv  Detail & Related papers  (2024-03-12T13:09:15Z)
• DPP-Based Adversarial Prompt Searching for Lanugage Models [56.73828162194457]
  Auto-regressive Selective Replacement Ascent (ASRA) is a discrete optimization algorithm that selects prompts based on both quality and similarity with determinantal point process (DPP) Experimental results on six different pre-trained language models demonstrate the efficacy of ASRA for eliciting toxic content.
  arXiv  Detail & Related papers  (2024-03-01T05:28:06Z)
• From Identifiable Causal Representations to Controllable Counterfactual Generation: A Survey on Causal Generative Modeling [17.074858228123706]
  We focus on fundamental theory, methodology, drawbacks, datasets, and metrics. We cover applications of causal generative models in fairness, privacy, out-of-distribution generalization, precision medicine, and biological sciences.
  arXiv  Detail & Related papers  (2023-10-17T05:45:32Z)
• RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model [93.8067369210696]
  Text-to-image generation (TTI) refers to the usage of models that could process text input and generate high fidelity images based on text descriptions. Diffusion models are one prominent type of generative model used for the generation of images through the systematic introduction of noises with repeating steps. In the era of large models, scaling up model size and the integration with large language models have further improved the performance of TTI models.
  arXiv  Detail & Related papers  (2023-09-02T03:27:20Z)
• Self-Consuming Generative Models Go MAD [21.056900382589266]
  We study how to use synthetic data to train generative AI algorithms for imagery, text, and other data types. Without enough fresh real data in each generation of an autophagous loop, future generative models are doomed to have their quality (precision) or diversity (recall) progressively decrease. We term this condition Model Autophagy Disorder (MAD), making analogy to mad cow disease.
  arXiv  Detail & Related papers  (2023-07-04T17:59:31Z)
• CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing [139.77117915309023]
  CRITIC allows large language models to validate and amend their own outputs in a manner similar to human interaction with tools. Comprehensive evaluations involving free-form question answering, mathematical program synthesis, and toxicity reduction demonstrate that CRITIC consistently enhances the performance of LLMs.
  arXiv  Detail & Related papers  (2023-05-19T15:19:44Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)
• Model Criticism for Long-Form Text Generation [113.13900836015122]
  We apply a statistical tool, model criticism in latent space, to evaluate the high-level structure of generated text. We perform experiments on three representative aspects of high-level discourse -- coherence, coreference, and topicality. We find that transformer-based language models are able to capture topical structures but have a harder time maintaining structural coherence or modeling coreference.
  arXiv  Detail & Related papers  (2022-10-16T04:35:58Z)
• Diversity vs. Recognizability: Human-like generalization in one-shot generative models [5.964436882344729]
  We propose a new framework to evaluate one-shot generative models along two axes: sample recognizability vs. diversity. We first show that GAN-like and VAE-like models fall on opposite ends of the diversity-recognizability space. In contrast, disentanglement transports the model along a parabolic curve that could be used to maximize recognizability.
  arXiv  Detail & Related papers  (2022-05-20T13:17:08Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• Topical Language Generation using Transformers [4.795530213347874]
  This paper presents a novel approach for Topical Language Generation (TLG) by combining a pre-trained LM with topic modeling information. We extend our model by introducing new parameters and functions to influence the quantity of the topical features presented in the generated text. Our experimental results demonstrate that our model outperforms the state-of-the-art results on coherency, diversity, and fluency while being faster in decoding.
  arXiv  Detail & Related papers  (2021-03-11T03:45:24Z)
• How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models [95.8037674226622]
  We introduce a 3-dimensional evaluation metric that characterizes the fidelity, diversity and generalization performance of any generative model in a domain-agnostic fashion. Our metric unifies statistical divergence measures with precision-recall analysis, enabling sample- and distribution-level diagnoses of model fidelity and diversity.
  arXiv  Detail & Related papers  (2021-02-17T18:25:30Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)
• Firearm Detection via Convolutional Neural Networks: Comparing a Semantic Segmentation Model Against End-to-End Solutions [68.8204255655161]
  Threat detection of weapons and aggressive behavior from live video can be used for rapid detection and prevention of potentially deadly incidents. One way for achieving this is through the use of artificial intelligence and, in particular, machine learning for image analysis. We compare a traditional monolithic end-to-end deep learning model and a previously proposed model based on an ensemble of simpler neural networks detecting fire-weapons via semantic segmentation.
  arXiv  Detail & Related papers  (2020-12-17T15:19:29Z)
• On the Transferability of Adversarial Attacksagainst Neural Text Classifier [121.6758865857686]
  We investigate the transferability of adversarial examples for text classification models. We propose a genetic algorithm to find an ensemble of models that can induce adversarial examples to fool almost all existing models. We derive word replacement rules that can be used for model diagnostics from these adversarial examples.
  arXiv  Detail & Related papers  (2020-11-17T10:45:05Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)
• Multi-Fact Correction in Abstractive Text Summarization [98.27031108197944]
  Span-Fact is a suite of two factual correction models that leverages knowledge learned from question answering models to make corrections in system-generated summaries via span selection. Our models employ single or multi-masking strategies to either iteratively or auto-regressively replace entities in order to ensure semantic consistency w.r.t. the source text. Experiments show that our models significantly boost the factual consistency of system-generated summaries without sacrificing summary quality in terms of both automatic metrics and human evaluation.
  arXiv  Detail & Related papers  (2020-10-06T02:51:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.