Closed-Loop LLM Discovery of Non-Standard Channel Priors in Vision Models

• URL: http://arxiv.org/abs/2601.08517v1
• Date: Tue, 13 Jan 2026 13:00:30 GMT
• Title: Closed-Loop LLM Discovery of Non-Standard Channel Priors in Vision Models
• Authors: Tolgay Atinc Uzun, Dmitry Ignatov, Radu Timofte,
• Abstract summary: Large Language Models (LLMs) offer a transformative approach to Neural Architecture Search (NAS)<n>We formulate the search as a sequence of conditional code generation tasks, where an LLM refines architectural specifications based on performance telemetry.<n>We generate a vast corpus of valid, shape-consistent architectures via Abstract Syntax Tree (AST) mutations.<n> Experimental results on CIFAR-100 validate the efficacy of this approach, demonstrating that the model yields statistically significant improvements in accuracy.
• Score: 48.83701310501069
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Channel configuration search the optimization of layer specifications such as layer widths in deep neural networks presents a complex combinatorial challenge constrained by tensor shape compatibility and computational budgets. We posit that Large Language Models (LLMs) offer a transformative approach to Neural Architecture Search (NAS), capable of reasoning about architectural code structure in ways that traditional heuristics cannot. In this paper, we investigate the application of an LLM-driven NAS framework to the problem of channel configuration. We formulate the search as a sequence of conditional code generation tasks, where an LLM refines architectural specifications based on performance telemetry. Crucially, we address the data scarcity problem by generating a vast corpus of valid, shape-consistent architectures via Abstract Syntax Tree (AST) mutations. While these mutated networks are not necessarily high-performing, they provide the critical volume of structural data required for the LLM to learn the latent relationship between channel configurations and model performance. This allows the LLM to internalize complex design patterns and apply them to optimize feature extraction strategies. Experimental results on CIFAR-100 validate the efficacy of this approach, demonstrating that the model yields statistically significant improvements in accuracy. Our analysis confirms that the LLM successfully acquires domain-specific architectural priors, distinguishing this method from random search and highlighting the immense potential of language-driven design in deep learning.

Related papers

• Cognitive-YOLO: LLM-Driven Architecture Synthesis from First Principles of Data for Object Detection [3.5554162308775408]
  We propose Cognitive-YOLO, a novel framework for Large Language Models (LLMs)-driven architecture synthesis.<n>Our method consists of three stages: first, an analysis module extracts key meta-features from the target dataset.<n>Second, the LLM reasons upon these features, augmented with state-of-the-art components retrieved via Retrieval-Augmented Generation (RAG), to synthesize the architecture into a structured Neural Architecture Description Language (NADL)<n>Third, a compiler instantiates this description into a deployable model.
  arXiv  Detail & Related papers  (2025-12-13T10:52:54Z)
• Improving LLM-based Ontology Matching with fine-tuning on synthetic data [0.0]
  Large Language Models (LLMs) are increasingly being integrated into various components of Ontology Matching pipelines.<n>This paper investigates the capability of LLMs to perform ontology matching directly on ontology modules and generate the corresponding alignments.<n>A dedicated fine-tuning strategy can enhance the model's matching performance in a zero-shot setting.
  arXiv  Detail & Related papers  (2025-11-27T16:46:45Z)
• LM-Searcher: Cross-domain Neural Architecture Search with LLMs via Unified Numerical Encoding [55.5535016040221]
  LM-Searcher is a novel framework for cross-domain neural architecture optimization.<n>Central to our approach is NCode, a universal numerical string representation for neural architectures.<n>Our dataset, encompassing a wide range of architecture-performance pairs, encourages robust and transferable learning.
  arXiv  Detail & Related papers  (2025-09-06T09:26:39Z)
• Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey [69.45421620616486]
  This work presents the first structured taxonomy and analysis of discrete tokenization methods designed for large language models (LLMs)<n>We categorize 8 representative VQ variants that span classical and modern paradigms and analyze their algorithmic principles, training dynamics, and integration challenges with LLM pipelines.<n>We identify key challenges including codebook collapse, unstable gradient estimation, and modality-specific encoding constraints.
  arXiv  Detail & Related papers  (2025-07-21T10:52:14Z)
• SAFT: Structure-Aware Fine-Tuning of LLMs for AMR-to-Text Generation [50.277959544420455]
  SAFT is a structure-aware fine-tuning approach that injects graph topology into pretrained language models.<n>We compute direction-sensitive positional encodings from the magnetic Laplacian of transformed AMRs.<n> SAFT sets a new state-of-the-art on AMR 3.0 with a 3.5 BLEU improvement over baselines.
  arXiv  Detail & Related papers  (2025-07-15T18:12:57Z)
• Enhancing Large Language Models through Structured Reasoning [15.472375478049823]
  We introduce a novel approach to enhance Large Language Models (LLMs) through explicit structured reasoning.<n>First, we convert unstructured data into structured formats by explicitly annotating reasoning steps.<n>We then employ this structured dataset to train LLMs through Supervised Fine-Tuning (SFT)
  arXiv  Detail & Related papers  (2025-06-25T08:36:12Z)
• Large Language Models are Good Relational Learners [55.40941576497973]
  We introduce Rel-LLM, a novel architecture that utilizes a graph neural network (GNN)- based encoder to generate structured relational prompts for large language models (LLMs)<n>Unlike traditional text-based serialization approaches, our method preserves the inherent relational structure of databases while enabling LLMs to process and reason over complex entity relationships.
  arXiv  Detail & Related papers  (2025-06-06T04:07:55Z)
• LLM-Guided Evolution: An Autonomous Model Optimization for Object Detection [0.0]
  In machine learning, Neural Architecture Search (NAS) requires domain knowledge of model design and a large amount of trial-and-error to achieve promising performance.<n>The Large Language Model (LLM)-Guided Evolution (GE) framework transformed this approach by incorporating LLMs to directly modify model source code for image classification algorithms on CIFAR data.<n>We show that LLM-GE produced variants with significant performance improvements, such as an increase in Mean Average Precision from 92.5% to 94.5%.
  arXiv  Detail & Related papers  (2025-04-03T05:06:06Z)
• Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models [79.41139393080736]
  Large language models (LLMs) have rapidly advanced and demonstrated impressive capabilities. In-Context Learning (ICL) and. Efficient Fine-Tuning (PEFT) are currently two mainstream methods for augmenting. LLMs to downstream tasks. We propose Reference Trustable Decoding (RTD), a paradigm that allows models to quickly adapt to new tasks without fine-tuning.
  arXiv  Detail & Related papers  (2024-09-30T10:48:20Z)
• Search for Efficient Large Language Models [52.98684997131108]
  Large Language Models (LLMs) have long held sway in the realms of artificial intelligence research. Weight pruning, quantization, and distillation have been embraced to compress LLMs, targeting memory reduction and inference acceleration. Most model compression techniques concentrate on weight optimization, overlooking the exploration of optimal architectures.
  arXiv  Detail & Related papers  (2024-09-25T21:32:12Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.