Cross-Stack Validation of Language-Model Training: A Clinical Fine-Tuning Case Study

• URL: http://arxiv.org/abs/2608.24267v1
• Date: Tue, 25 Aug 2026 08:55:45 GMT
• Title: Cross-Stack Validation of Language-Model Training: A Clinical Fine-Tuning Case Study
• Abstract summary: We study whether independently implemented training stacks can serve as differential oracles for a whole fine-tuning pipeline.<n>We apply it to a LoRA adaptation of Qwen3-0.6B over 168,574 clinical question-answer pairs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural network training has an oracle problem: a run can converge normally and yield a usable model while the software beneath it computes something other than specified. Almost all such work runs on one stack, so there is rarely anything independent to check against. We study whether independently implemented training stacks can serve as differential oracles for a whole fine-tuning pipeline, rather than the operators and inference paths that prior differential testing targets. We define a trajectory-level protocol -- a shared specification, cross-check points spanning arithmetic, model loading, data rendering and the learning trajectory, and a separation of independence of the stack, the orchestration and the language runtime -- and apply it to a LoRA adaptation of Qwen3-0.6B over 168,574 clinical question-answer pairs under PyTorch and under numbat, an independent framework written in Zig, driven natively and through its C interface from six languages. Across 42 paired evaluations spanning a full epoch the two stacks' held-out cross-entropy differs by 0.134% on average, and four implementations end the epoch within 0.15% of one another. The comparison exposed 17 faults that single-implementation development had missed, two of them notable for software engineering. The fault with the largest effect on the trained model lay outside the numerical kernels: a mismatch in how clinical text was rendered moved held-out loss 0.15, some 500 times more than the arithmetic faults found beside it. And four faults were reachable only from a language whose memory model differs from the first two implementations: a scheduler migrating work across threads, a collector blind to device memory, an ownership discipline needing a primitive the interface lacked. Implementation diversity has several axes, and the runtime is one.

Related papers

• Numbat: Building and Verifying a Self-Contained Machine-Learning Stack [0.0]
  numbat is a machine-learning stack written in one general-purpose language (Zig) with no third-party dependencies.<n>We report on the construction and verification of numbat.
  arXiv  Detail & Related papers  (2026-09-09T08:36:12Z)
• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• Kozuchi Agent: A Language-Agnostic Open-Weight Agent for Software Repair [13.668549452625117]
  Kozuchi Agent is a language-agnostic open-weight repair agent and CI-operated evaluation pipeline.<n>Explicit phases, persistent state, deterministic tools, a model-independent action interface, and cross-agent test-time selection make runs auditable and repeatable.
  arXiv  Detail & Related papers  (2026-08-16T07:06:37Z)
• Differentiate the Evaluator, Not the Program: An Efficient Runtime Representation for Neuro-Symbolic Learning [0.0]
  We present the Native Differentiable Virtual Machine (NDVM), a runtime representation that differentiates executable programs without compiling each candidate into a separate graph.<n>NDVM separates symbolic structure from differentiable numeric state: tags, symbols, environments, control remain native runtime data, while numeric payloads live in dense batched buffers with exact reverse-mode gradients recorded along the realized execution trace.<n>We realize NDVM as a native runtime with forward and gradient equivalence to the backend reference Scheme, about 60x per-lane batch amortization, near-linear multicore scaling, and two independent front ends.
  arXiv  Detail & Related papers  (2026-07-03T19:29:35Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• LIDARLearn: A Unified Deep Learning Library for 3D Point Cloud Classification, Segmentation, and Self-Supervised Representation Learning [1.2599533416395765]
  Three-dimensional (3D) point cloud analysis has become central to applications ranging from autonomous driving and robotics to forestry and ecological monitoring.<n>lib is a unified, PyTorch library that integrates over 55 model configurations covering 29 supervised architectures, seven SSL pre-training methods, and five PEFT strategies.<n>lib provides standardised training runners, cross-validation with stratified $K$-fold splitting, automated/CSV table generation, built-in Friedman/Nemenyi statistical testing with critical-difference diagrams for rigorous multi-model comparison, and a comprehensive test suite with 2,200+ automated tests
  arXiv  Detail & Related papers  (2026-04-12T19:10:12Z)
• AlphaLab: Autonomous Multi-Agent Research Across Optimization Domains with Frontier LLMs [14.167136550376497]
  We present AlphaLab, an autonomous research harness that automates the full experimental cycle in computation-intensive domains.<n>It adapts to the domain and explores the data, writing analysis code and producing a research report.<n>It runs large-scale experiments via a Strategist/Worker loop, accumulating domain knowledge in a persistent playbook.
  arXiv  Detail & Related papers  (2026-03-31T21:16:20Z)
• IT$^3$: Idempotent Test-Time Training [95.78053599609044]
  Deep learning models often struggle when deployed in real-world settings due to distribution shifts between training and test data.<n>We present Idempotent Test-Time Training (IT$3$), a novel approach that enables on-the-fly adaptation to distribution shifts using only the current test instance.<n>Our results suggest that idempotence provides a universal principle for test-time adaptation that generalizes across domains and architectures.
  arXiv  Detail & Related papers  (2024-10-05T15:39:51Z)
• Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared Randomness [86.61582747039053]
  Language model training in distributed settings is limited by the communication cost of exchanges. We extend recent work using shared randomness to perform distributed fine-tuning with low bandwidth.
  arXiv  Detail & Related papers  (2023-06-16T17:59:51Z)
• Simple Stochastic and Online Gradient DescentAlgorithms for Pairwise Learning [65.54757265434465]
  Pairwise learning refers to learning tasks where the loss function depends on a pair instances. Online descent (OGD) is a popular approach to handle streaming data in pairwise learning. In this paper, we propose simple and online descent to methods for pairwise learning.
  arXiv  Detail & Related papers  (2021-11-23T18:10:48Z)
• A Unified Framework of Surrogate Loss by Refactoring and Interpolation [65.60014616444623]
  We introduce UniLoss, a unified framework to generate surrogate losses for training deep networks with gradient descent. We validate the effectiveness of UniLoss on three tasks and four datasets.
  arXiv  Detail & Related papers  (2020-07-27T21:16:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.