Continually self-improving AI

• URL: http://arxiv.org/abs/2603.18073v1
• Date: Wed, 18 Mar 2026 04:54:05 GMT
• Title: Continually self-improving AI
• Abstract summary: Modern language model-based AI systems are remarkably powerful, yet their capabilities remain fundamentally capped by their human creators.<n>This thesis takes a small step toward overcoming these inherent limitations, presenting three chapters aimed at breaking these dependencies to create continually self-improving AI.<n>First, to overcome this data-efficiency barrier in knowledge acquisition, we propose a synthetic data approach that diversifies and amplifies small corpora into rich knowledge representations.<n>Second, to reduce reliance on human data, we show that given a fixed amount of such data, the model can self-generate synthetic data to bootstrap its fundamental pretraining capabilities.
• Score: 5.390995889911029
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern language model-based AI systems are remarkably powerful, yet their capabilities remain fundamentally capped by their human creators in three key ways. First, although a model's weights can be updated via fine-tuning, acquiring new knowledge from small, specialized corpora after pretraining remains highly data-inefficient. Second, the training of these systems relies heavily on finite, human-generated data from across history. Third, the pipelines used to train AI models are confined by the algorithms that human researchers can discover and explore. This thesis takes a small step toward overcoming these inherent limitations, presenting three chapters aimed at breaking these dependencies to create continually self-improving AI. First, to overcome this data-efficiency barrier in knowledge acquisition, we propose a synthetic data approach that diversifies and amplifies small corpora into rich knowledge representations, enabling a model to effectively update its parameters from limited source material. Second, to reduce reliance on human data, we show that given a fixed amount of such data, the model can self-generate synthetic data to bootstrap its fundamental pretraining capabilities without distillation from any off-the-shelf, instruction-tuned LM. Finally, to transcend human-engineered training paradigms, we demonstrate that by scaling search during test time over the space of algorithms, AI can search over a larger space of learning algorithm configurations than human researchers can explore manually.

Related papers

• The FIL Hypothesis: Inductive Biases Help with Kernel Engineering [88.25617363280156]
  General-purpose methods that scale with computation and data ultimately outperform those with built-in human knowledge.<n>We revisit this principle by observing a new and critical scaling dimension: the duration of the Feedback Information Loop (FIL)<n>We argue that future AI applications in science and the physical world will inherently involve FILs ranging from hours to weeks.
  arXiv  Detail & Related papers  (2026-06-29T15:16:33Z)
• HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining [93.5192770515694]
  Embodied foundation models are expected to benefit from data scaling like large language models, but face a much tighter data bottleneck.<n>Teleoperated real-robot trajectories remain the dominant pretraining source due to their precise action supervision and embodiment alignment.<n>Embodied foundation models pretrained on egocentric data achieve a 24% lower validation loss on real-robot action prediction.
  arXiv  Detail & Related papers  (2026-06-18T17:37:34Z)
• Meta-Learning and Synthetic Data for Automated Pretraining and Finetuning [2.657867981416885]
  Growing number of pretrained models in Machine Learning (ML) presents significant challenges for practitioners.<n>As models grow in scale, the increasing reliance on real-world data poses a bottleneck for training and requires leveraging data more effectively.<n>This dissertation adopts meta-learning to extend automated machine learning to the deep learning domain.
  arXiv  Detail & Related papers  (2025-06-11T12:48:45Z)
• Shifting AI Efficiency From Model-Centric to Data-Centric Compression [67.45087283924732]
  We argue that the focus of research for AI is shifting from model-centric compression to data-centric compression.<n>Data-centric compression improves AI efficiency by directly compressing the volume of data processed during model training or inference.<n>Our work aims to provide a novel perspective on AI efficiency, synthesize existing efforts, and catalyze innovation to address the challenges posed by ever-increasing context lengths.
  arXiv  Detail & Related papers  (2025-05-25T13:51:17Z)
• Generalising from Self-Produced Data: Model Training Beyond Human Constraints [0.0]
  This paper introduces a novel framework in which AI models autonomously generate and validate new knowledge.<n>Central to this approach is an unbounded, ungamable numeric reward that guides learning without requiring human benchmarks.
  arXiv  Detail & Related papers  (2025-04-07T03:48:02Z)
• Learning by Surprise: Surplexity for Mitigating Model Collapse in Generative AI [1.6545633988217645]
  As synthetic content infiltrates the web, generative AI models may be retrained on their own outputs.<n>This leads to model collapse: a progressive loss of performance and diversity across generations.<n>We introduce new measures that characterise collapse directly from a model's next-token probability distributions.
  arXiv  Detail & Related papers  (2024-10-16T08:02:48Z)
• The Trifecta: Three simple techniques for training deeper Forward-Forward networks [0.0]
  We propose a collection of three techniques that synergize exceptionally well and drastically improve the Forward-Forward algorithm on deeper networks. Our experiments demonstrate that our models are on par with similarly structured, backpropagation-based models in both training speed and test accuracy on simple datasets.
  arXiv  Detail & Related papers  (2023-11-29T22:44:32Z)
• Exploration with Principles for Diverse AI Supervision [88.61687950039662]
  Training large transformers using next-token prediction has given rise to groundbreaking advancements in AI. While this generative AI approach has produced impressive results, it heavily leans on human supervision. This strong reliance on human oversight poses a significant hurdle to the advancement of AI innovation. We propose a novel paradigm termed Exploratory AI (EAI) aimed at autonomously generating high-quality training data.
  arXiv  Detail & Related papers  (2023-10-13T07:03:39Z)
• Ex-Model: Continual Learning from a Stream of Trained Models [12.27992745065497]
  We argue that continual learning systems should exploit the availability of compressed information in the form of trained models. We introduce and formalize a new paradigm named "Ex-Model Continual Learning" (ExML), where an agent learns from a sequence of previously trained models instead of raw data.
  arXiv  Detail & Related papers  (2021-12-13T09:46:16Z)
• Data-Driven and SE-assisted AI Model Signal-Awareness Enhancement and Introspection [61.571331422347875]
  We propose a data-driven approach to enhance models' signal-awareness. We combine the SE concept of code complexity with the AI technique of curriculum learning. We achieve up to 4.8x improvement in model signal awareness.
  arXiv  Detail & Related papers  (2021-11-10T17:58:18Z)
• Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedback [82.96694147237113]
  We present Skill Preferences, an algorithm that learns a model over human preferences and uses it to extract human-aligned skills from offline data. We show that SkiP enables a simulated kitchen robot to solve complex multi-step manipulation tasks.
  arXiv  Detail & Related papers  (2021-08-11T18:04:08Z)
• What Matters in Learning from Offline Human Demonstrations for Robot Manipulation [64.43440450794495]
  We conduct an extensive study of six offline learning algorithms for robot manipulation. Our study analyzes the most critical challenges when learning from offline human data. We highlight opportunities for learning from human datasets.
  arXiv  Detail & Related papers  (2021-08-06T20:48:30Z)
• Computational model discovery with reinforcement learning [3.005240085945858]
  The motivation of this study is to leverage recent breakthroughs in artificial intelligence research to unlock novel solutions to scientific problems encountered in computational science. To address the human intelligence limitations in discovering reduced-order models, we propose to supplement human thinking with artificial intelligence.
  arXiv  Detail & Related papers  (2019-12-29T22:56:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.