Thomson: Continual Learning of Frontier Models for SovereignAI

• URL: http://arxiv.org/abs/2608.27147v1
• Date: Thu, 27 Aug 2026 13:59:36 GMT
• Title: Thomson: Continual Learning of Frontier Models for SovereignAI
• Abstract summary: We argue that frontier performance is achievable by a wide range of institutions through Continual Learning on readily available open-weight models.<n>We demonstrate this with Thomson, a general-purpose frontier model trained with an enhanced focus on high-stakes professional work.
• Score: 20.382494818209253
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: The development of frontier models is commonly perceived to be the exclusive remit of a small number of heavily funded players, creating an information, economic and power asymmetry between developers and the diverse user base of modern AI. Recent public discourse acknowledges this concern, calling for SovereignAI (an organisation's capability to independently build, deploy and govern AI use), but offers little concrete advice on how this can be achieved in the short term under a diversity of funding settings. We argue that frontier performance is achievable by a wide range of institutions through Continual Learning on readily available open-weight models. Unlike limited approaches such as small-scale fine-tuning, prompt engineering, or tool-augmentation of a frozen model, our approach exploits a modern mid- & post-training stack while introducing safeguards that preserve both plasticity and stability at each stage, making the minimal number of high-impact interventions on the parameters. This yields gains comparable to those typically seen across multiple successive model generations, at compute and personnel budgets substantially lower than commonly thought, making ownership of large parts of the SovereignAI stack (model, tool infrastructure, values & data privacy) viable for far more actors. We demonstrate this with Thomson, a general-purpose frontier model trained with an enhanced focus on high-stakes professional work. Thomson performs competitively with recent frontier models across agentic tasks, safety, legal, tax & multilingualism, and large-scale Deep Research. Evaluations show a distinctive $π$-shaped pattern: distinct improvements across a wide range of capabilities, including those not explicitly targeted, while almost completely eliminating the forgetting problem common to narrow domain adaptation.

Related papers

• DeRelayL: Sustainable Decentralized Relay Learning [53.733457724691654]
  This paper proposes a novel training paradigm named decentralized relay learning (DeRelayL)<n>DeRelayL is a sustainable learning system where permissionless participants can contribute to model training in a relay-like manner and share the model.<n>In detail, this paper presents the architecture and workflow of DeRelayL, designs incentive mechanisms to ensure sustainability, and conducts theoretical analysis and numerical simulations to demonstrate its effectiveness.
  arXiv  Detail & Related papers  (2026-04-30T15:45:30Z)
• EoS-FM: Can an Ensemble of Specialist Models act as a Generalist Feature Extractor? [8.178030486012437]
  We present an Ensemble-of-Specialists framework for building Remote Sensing Foundation Models (RSFMs)<n>Our method decomposes the training process into lightweight, task-specific ConvNeXtV2 specialists that can be frozen and reused.<n>Our framework sets a new direction for building scalable and efficient RSFMs.
  arXiv  Detail & Related papers  (2025-11-26T15:52:56Z)
• Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs [80.72350166388601]
  Nemotron Elastic is a framework for building reasoning-oriented LLMs.<n>It embeds nested submodels within a single parent model.<n>Each of these submodels shares weights with the parent model and can be extracted zero-shot during deployment.
  arXiv  Detail & Related papers  (2025-11-20T18:59:21Z)
• NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching [64.10695425442164]
  We introduce NExT-OMNI, an open-source omnimodal foundation model that achieves unified modeling through discrete flow paradigms.<n>Trained on large-scale interleaved text, image, video, and audio data, NExT-OMNI delivers competitive performance on multimodal generation and understanding benchmarks.<n>To advance further research, we release training details, data protocols, and open-source both the code and model checkpoints.
  arXiv  Detail & Related papers  (2025-10-15T16:25:18Z)
• Revealing the Power of Post-Training for Small Language Models via Knowledge Distillation [43.68215777330875]
  We introduce a systematic post-training pipeline that efficiently enhances small model accuracy.<n>The resulting instruction-tuned model achieves state-of-the-art performance.<n>This work provides a practical and efficient solution for developing high-performance language models on Ascend edge devices.
  arXiv  Detail & Related papers  (2025-09-30T16:40:55Z)
• Multi-Model Synthetic Training for Mission-Critical Small Language Models [0.0]
  We present a novel approach that achieves a 261x cost reduction for maritime intelligence.<n>Our method transforms 3.2 billion Automatic Identification System (AIS) vessel tracking records into 21,543 synthetic question and answer pairs.<n>The resulting fine-tuned Qwen2.5-7B model achieves 75% accuracy on maritime tasks, while being substantially cheaper than using a larger model for inference.
  arXiv  Detail & Related papers  (2025-09-16T13:04:48Z)
• AI in a vat: Fundamental limits of efficient world modelling for agent sandboxing and interpretability [84.52205243353761]
  Recent work proposes using world models to generate controlled virtual environments in which AI agents can be tested before deployment.<n>We investigate ways of simplifying world models that remain agnostic to the AI agent under evaluation.
  arXiv  Detail & Related papers  (2025-04-06T20:35:44Z)
• A Framework for Situating Innovations, Opportunities, and Challenges in Advancing Vertical Systems with Large AI Models [30.35173355997027]
  Large AI models are deployed in high-stakes verticals such as healthcare, education, and law.<n>These challenges necessitate cross-disciplinary innovations to align the models' capabilities with the needs of real-world applications.<n>We introduce a framework that addresses this gap through a layer-wise abstraction of innovations aimed at meeting users' requirements.
  arXiv  Detail & Related papers  (2025-04-03T17:40:11Z)
• Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners [85.03486419424647]
  KnowNo is a framework for measuring and aligning the uncertainty of large language models. KnowNo builds on the theory of conformal prediction to provide statistical guarantees on task completion.
  arXiv  Detail & Related papers  (2023-07-04T21:25:12Z)
• eP-ALM: Efficient Perceptual Augmentation of Language Models [70.47962271121389]
  We propose to direct effort to efficient adaptations of existing models, and propose to augment Language Models with perception. Existing approaches for adapting pretrained models for vision-language tasks still rely on several key components that hinder their efficiency. We show that by freezing more than 99% of total parameters, training only one linear projection layer, and prepending only one trainable token, our approach (dubbed eP-ALM) significantly outperforms other baselines on VQA and Captioning.
  arXiv  Detail & Related papers  (2023-03-20T19:20:34Z)
• Legal-Tech Open Diaries: Lesson learned on how to develop and deploy light-weight models in the era of humongous Language Models [10.086015702323971]
  We follow the steps of the R&D group of a modern legal-tech start-up and present important insights on model development and deployment. We start from ground zero by pre-training multiple domain-specific multi-lingual LMs which are a better fit to contractual and regulatory text. We present benchmark results of such models in a half-public half-private legal benchmark comprising 5 downstream tasks showing the impact of larger model size.
  arXiv  Detail & Related papers  (2022-10-24T10:08:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.