A Survey on Self-Evolution of Large Language Models

• URL: http://arxiv.org/abs/2404.14387v2
• Date: Mon, 3 Jun 2024 17:47:30 GMT
• Title: A Survey on Self-Evolution of Large Language Models
• Authors: Zhengwei Tao, Ting-En Lin, Xiancai Chen, Hangyu Li, Yuchuan Wu, Yongbin Li, Zhi Jin, Fei Huang, Dacheng Tao, Jingren Zhou,
• Abstract summary: Large language models (LLMs) have significantly advanced in various fields and intelligent agent applications. To address this issue, self-evolution approaches that enable LLMs to autonomously acquire, refine, and learn from experiences generated by the model itself are rapidly growing.
• Score: 116.54238664264928
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) have significantly advanced in various fields and intelligent agent applications. However, current LLMs that learn from human or external model supervision are costly and may face performance ceilings as task complexity and diversity increase. To address this issue, self-evolution approaches that enable LLM to autonomously acquire, refine, and learn from experiences generated by the model itself are rapidly growing. This new training paradigm inspired by the human experiential learning process offers the potential to scale LLMs towards superintelligence. In this work, we present a comprehensive survey of self-evolution approaches in LLMs. We first propose a conceptual framework for self-evolution and outline the evolving process as iterative cycles composed of four phases: experience acquisition, experience refinement, updating, and evaluation. Second, we categorize the evolution objectives of LLMs and LLM-based agents; then, we summarize the literature and provide taxonomy and insights for each module. Lastly, we pinpoint existing challenges and propose future directions to improve self-evolution frameworks, equipping researchers with critical insights to fast-track the development of self-evolving LLMs. Our corresponding GitHub repository is available at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/Awesome-Self-Evolution-of-LLM

Related papers

• Long Term Memory: The Foundation of AI Self-Evolution [48.52678410533424]
  Large language models (LLMs) like GPTs, trained on vast datasets, have demonstrated impressive capabilities in language understanding, reasoning, and planning. Most studies focus on enhancing these models by training on ever-larger datasets to build more powerful foundation models. Unlike large-scale training, enabling models to evolve during inference is equally crucial, a process we refer to as AI self-evolution.
  arXiv  Detail & Related papers  (2024-10-21T06:09:30Z)
• Towards more realistic evaluation of LLM-based code generation: an experimental study and beyond [36.1669124651617]
  We conduct an empirical study to understand Large Language Models' code generation performance within settings that reflect the evolving nature of software development. We find that previous evolving-ignored evaluation approaches lead to inflated performance of the LLMs, ranging from 10.0% to 61.1%.
  arXiv  Detail & Related papers  (2024-06-11T03:19:18Z)
• Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing [56.75702900542643]
  We introduce AlphaLLM for the self-improvements of Large Language Models. It integrates Monte Carlo Tree Search (MCTS) with LLMs to establish a self-improving loop. Our experimental results show that AlphaLLM significantly enhances the performance of LLMs without additional annotations.
  arXiv  Detail & Related papers  (2024-04-18T15:21:34Z)
• Bias Amplification in Language Model Evolution: An Iterated Learning Perspective [27.63295869974611]
  We draw parallels between the behavior of Large Language Models (LLMs) and the evolution of human culture. Our approach involves leveraging Iterated Learning (IL), a Bayesian framework that elucidates how subtle biases are magnified during human cultural evolution. This paper outlines key characteristics of agents' behavior in the Bayesian-IL framework, including predictions that are supported by experimental verification.
  arXiv  Detail & Related papers  (2024-04-04T02:01:25Z)
• LLM Guided Evolution - The Automation of Models Advancing Models [0.0]
  "Guided Evolution" (GE) is a novel framework that diverges from traditional machine learning approaches. "Evolution of Thought" (EoT) enhances GE by enabling LLMs to reflect on and learn from the outcomes of previous mutations. Our application of GE in evolving the ExquisiteNetV2 model demonstrates its efficacy.
  arXiv  Detail & Related papers  (2024-03-18T03:44:55Z)
• Continual Learning for Large Language Models: A Survey [95.79977915131145]
  Large language models (LLMs) are not amenable to frequent re-training, due to high training costs arising from their massive scale. This paper surveys recent works on continual learning for LLMs.
  arXiv  Detail & Related papers  (2024-02-02T12:34:09Z)
• Supervised Knowledge Makes Large Language Models Better In-context Learners [94.89301696512776]
  Large Language Models (LLMs) exhibit emerging in-context learning abilities through prompt engineering. The challenge of improving the generalizability and factuality of LLMs in natural language understanding and question answering remains under-explored. We propose a framework that enhances the reliability of LLMs as it: 1) generalizes out-of-distribution data, 2) elucidates how LLMs benefit from discriminative models, and 3) minimizes hallucinations in generative tasks.
  arXiv  Detail & Related papers  (2023-12-26T07:24:46Z)
• SELF: Self-Evolution with Language Feedback [68.6673019284853]
  'SELF' (Self-Evolution with Language Feedback) is a novel approach to advance large language models. It enables LLMs to self-improve through self-reflection, akin to human learning processes. Our experiments in mathematics and general tasks demonstrate that SELF can enhance the capabilities of LLMs without human intervention.
  arXiv  Detail & Related papers  (2023-10-01T00:52:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.