Similarity-Guided Curriculum Fine-Tuning of LLMs for Neural Architecture Synthesis

• URL: http://arxiv.org/abs/2607.11591v1
• Date: Mon, 13 Jul 2026 14:14:30 GMT
• Title: Similarity-Guided Curriculum Fine-Tuning of LLMs for Neural Architecture Synthesis
• Abstract summary: We introduce a MinHash-based similarity scheduling framework that constructs a progressive curriculum over neural architecture code for neural architecture search (NAS)<n>We evaluate the framework on OlympicCoder-7B within the LEMUR benchmark on CIFAR-10 image classification.<n>We report a cross-dataset transfer observation on SVHN, where direct base-model generation without curriculum yields 27% peak SR at substantially lower accuracy (60.5%) than the CIFAR-10 equivalent.
• Score: 48.83701310501069
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Introduce a MinHash-based similarity scheduling framework that constructs a progressive curriculum over neural architecture code for LLM-based neural architecture search (NAS). Using 128-permutation MinHash signatures over normalised 7-gram source code shingles, we partition the reference pool into similarity bands and present them in increasing architectural heterogeneity, with the best LoRA adapter from each stage merged cumulatively into the backbone. We evaluate the framework on OlympicCoder-7B within the LEMUR benchmark on CIFAR-10 image classification, generating N =15 candidate architectures per epoch across six progressive fine-tuning steps. The curriculum achieves 60% peak success rate at the high-similarity level without post-processing repair. A 2*2 ablation at the most diverse level curriculum versus base model, with versus without partial interface repair reveals that without repair the base model (47% peak SR) substantially outperforms the curriculum model (7% SR), while adding partial repair brings both to 53% SR. This pattern is consistent with merge-level weight drift progressively erasing evaluator-interface priors, and suggests that interface repair and curriculum scheduling target distinct failure modes. We further report a cross-dataset transfer observation on SVHN, where direct base-model generation without curriculum warmup yields 27% peak SR at substantially lower accuracy (60.5%) than the CIFAR-10 equivalent, consistent with the increased synthesis difficulty of the unq-family anchor architecture.

Related papers

• ISO: An RLVR-Native Optimization Stack [50.40395312897848]
  We operationalize spectral inheritance as Isospectral Optimization (ISO)<n>Offline, ISO-Merger combines the frame changes of shared-base specialists into a single fixed-spectrum model.<n>Online, ISO-Merger applies a chosen base, including AdamW and Muon, to the frame variables while keeping the base spectra fixed.
  arXiv  Detail & Related papers  (2026-07-21T17:51:36Z)
• Systematic Evaluation of Learning Rate Scheduling Strategies Across Heterogeneous Architectures [48.83701310501069]
  We investigate the impact of schedulers on classification accuracy across a diverse pool of architectures.<n>We applied 25 scheduler configurations across nine PyTorch families, evaluating a total of 3,938 model variants on CIFAR-10.<n>The results show that the choice of scheduler depends heavily on the architecture: CosineAnnealingWarmRestarts and CyclicLR consistently outperform basic decay strategies.
  arXiv  Detail & Related papers  (2026-07-09T14:06:39Z)
• DynaGraph: Lightweight Multi-Model Interaction Framework via Dynamic Topological Reconfiguration [7.0972889296550825]
  DynaGraph is a lightweight multi-model framework driven by dynamic topological reconfiguration.<n>It multiplexes time-division PEFT adapters over a shared base model, enabling both full system training and deployment.<n>It reduces latency by up to 68.1% and token consumption by 68.6% compared to unconstrained dynamic architectures.
  arXiv  Detail & Related papers  (2026-05-28T07:33:29Z)
• Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs [48.83701310501069]
  Large language models (LLMs) show strong potential for neural architecture generation.<n>Existing approaches produce complete model implementations from scratch.<n>We propose Delta-Code Generation, where fine-tuned LLMs generate compact unified diffs.
  arXiv  Detail & Related papers  (2026-05-06T13:32:05Z)
• Multi-Frequency Local Plasticity for Visual Representation Learning [0.40048696135519796]
  We study how far structured architectural bias can compensate for the absence of end-to-end gradient-based representation learning in visual recognition.<n>We introduce a modular hierarchical framework combining: (i) fixed multi-frequency Gabor decomposition into F=7 parallel streams; (ii) within-stream competitive learning with Hebbian and Oja updates and anti-Hebbian decorrelation; and (iii) an associative memory module inspired by modern Hopfield retrieval.
  arXiv  Detail & Related papers  (2026-04-09T18:30:47Z)
• From Memorization to Creativity: LLM as a Designer of Novel Neural-Architectures [48.83701310501069]
  Large language models (LLMs) excel in program synthesis, yet their ability to autonomously navigate neural architecture design-balancing reliability, performance, and structural novelty--remains underexplored.<n>We address this by placing a code-oriented LLM within a closed-loop synthesis framework, analyzing its evolution over 22 supervised fine-tuning cycles.
  arXiv  Detail & Related papers  (2026-01-06T13:20:28Z)
• Is Architectural Complexity Overrated? Competitive and Interpretable Knowledge Graph Completion with RelatE [6.959701672059059]
  RelatE is an interpretable and modular method that efficiently integrates dual representations for entities and relations.<n>It achieves competitive or superior performance on standard benchmarks.<n>Perturbation studies demonstrate improved robustness, with MRR reduced by up to 61% relative to TransE and by up to 19% compared to RotatE.
  arXiv  Detail & Related papers  (2025-05-25T04:36:52Z)
• Restore Anything Model via Efficient Degradation Adaptation [129.38475243424563]
  RAM takes a unified path that leverages inherent similarities across various degradations to enable efficient and comprehensive restoration.<n> RAM's SOTA performance confirms RAM's SOTA performance, reducing model complexity by approximately 82% in trainable parameters and 85% in FLOPs.
  arXiv  Detail & Related papers  (2024-07-18T10:26:53Z)
• Open-Set Recognition: A Good Closed-Set Classifier is All You Need [146.6814176602689]
  We show that the ability of a classifier to make the 'none-of-above' decision is highly correlated with its accuracy on the closed-set classes. We use this correlation to boost the performance of the cross-entropy OSR 'baseline' by improving its closed-set accuracy. We also construct new benchmarks which better respect the task of detecting semantic novelty.
  arXiv  Detail & Related papers  (2021-10-12T17:58:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.