Context-Instrumental Data Distillation for Kubernetes Manifest Generation: Method and Experimental Evaluation

• URL: http://arxiv.org/abs/2605.25835v1
• Date: Mon, 25 May 2026 13:30:38 GMT
• Title: Context-Instrumental Data Distillation for Kubernetes Manifest Generation: Method and Experimental Evaluation
• Abstract summary: This paper examines the specialization of Small Language Models with up to 4 billion parameters for generating artifacts in domain-specific languages (AML)<n>We propose the context-instrumental data distillation method: the source corpus is formed through synthetic generation and, in an extended scheme, through reverse instruction generation from real Y files.
• Score: 36.94429692322632
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper examines the specialization of Small Language Models (SLMs) with up to 4 billion parameters for generating artifacts in domain-specific languages (DSL). Kubernetes manifests are chosen as the target domain. We propose the context-instrumental data distillation method: the source corpus is formed through synthetic generation and, in an extended scheme, through reverse instruction generation from real Kubernetes YAML files, with pairs included in training only upon passing external validators and matching the domain context model. Unlike classical KL-divergence knowledge distillation, the baseline implementation reduces to supervised fine-tuning on instrumentally verified examples. The experimental section presents a pilot implementation under resource-constrained conditions: the DeepSeek-V4 Flash API serves as the teacher for synthetic generation, while Qwen2.5-Coder-1.5B-Instruct is fine-tuned via LoRA on CPU. On the K8s-Distill-Pilot corpus (train_1200, validation_100, test_200), we achieved full-pass@1 = 91.5% (183/200) with a stricter prompt formulation and max_new_tokens=768. The key empirical finding is that for Kubernetes YAML, result quality in the pilot depended more on strict output format requirements than on simply increasing the number of training examples.

Related papers

• Numbat: Building and Verifying a Self-Contained Machine-Learning Stack [0.0]
  numbat is a machine-learning stack written in one general-purpose language (Zig) with no third-party dependencies.<n>We report on the construction and verification of numbat.
  arXiv  Detail & Related papers  (2026-09-09T08:36:12Z)
• Prototype Language Models [32.76699413390825]
  Prototype language models form each prediction via a sparse, non-negative mixture of learned prototypes.<n>We show that sparse prototype structure localizes curvature in the loss landscape, yielding a more tractable Hessian.
  arXiv  Detail & Related papers  (2026-07-01T06:45:02Z)
• SIEVE: Sample-Efficient Parametric Learning from Natural Language [66.46967795931559]
  We propose SIEVE, a method for sample-efficient parametric learning from natural language context.<n>We use a novel synthetic data generation pipeline, SIEVE-GEN, that leverages the insight that context is decomposable.<n>Our results show that SIEVE outperforms prior context distillation methods using just three query examples.
  arXiv  Detail & Related papers  (2026-02-02T18:52:52Z)
• Adaptation of Embedding Models to Financial Filings via LLM Distillation [10.744318713371383]
  This paper introduces a scalable pipeline that trains specialized models from an unlabeled corpus using a general purpose retrieval embedding model as foundation.<n>Our method yields an average of 27.7% improvement in MRR$textt@$5, 44.6% improvement in mean DCG$textt@$5 across 14 financial filing types measured over 21,800 query-document pairs.
  arXiv  Detail & Related papers  (2025-12-08T22:43:14Z)
• LLM as a Neural Architect: Controlled Generation of Image Captioning Models Under Strict API Contracts [48.83701310501069]
  We present NN-Caption, an LLM-guided neural architecture search pipeline.<n>It generates runnable image-captioning models by composing CNN encoders from LEMUR's classification backbones.<n>This work presents a pipeline that integrates prompt-based code generation with automatic evaluation.
  arXiv  Detail & Related papers  (2025-12-07T10:47:28Z)
• David vs. Goliath: A comparative study of different-sized LLMs for code generation in the domain of automotive scenario generation [1.6752458252726459]
  NL-to-Scenic generation with large language models (LLMs) suffers from scarce data, limited metrics.<n>We introduce NL2Scenic, an open dataset and framework with 146 NL/Scenic pairs, a difficulty-stratified 30-case test split, and an Example Retriever.<n>We evaluate 13 models: four proprietary (GPT-4o, GPT-5, Claude-Sonnet-4, Gemini-2.5-pro) and nine open-source code models (Qwen2.5Coder 0.5B-32B; CodeLlama 7B/13B/34B)
  arXiv  Detail & Related papers  (2025-10-15T21:37:02Z)
• The Hidden DNA of LLM-Generated JavaScript: Structural Patterns Enable High-Accuracy Authorship Attribution [2.334824705384299]
  We present the first large-scale study exploring whether JavaScript code generated by Large Language Models can reveal which model produced it.<n>We show that individual LLMs leave unique stylistic signatures, even among models belonging to the same family or parameter size.<n>We introduce LLM-NodeJS, a dataset of 50,000 Node.js back-end programs from 20 large language models.
  arXiv  Detail & Related papers  (2025-10-12T07:51:03Z)
• DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation [68.19756761027351]
  Diffusion large language models (dLLMs) are compelling alternatives to autoregressive (AR) models.<n>We investigate their denoising processes and reinforcement learning methods.<n>Our work provides deeper insight into the machinery of dLLM generation and offers an effective, diffusion-native RL training framework.
  arXiv  Detail & Related papers  (2025-06-25T17:35:47Z)
• Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generation [0.0]
  We analyze the behaviors of open large language models (LLMs) on the task of data-to-text (D2T) generation. We find that open LLMs can generate fluent and coherent texts in zero-shot settings from data in common formats collected with Quintd.
  arXiv  Detail & Related papers  (2024-01-18T18:15:46Z)
• The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute [66.84421705029624]
  We introduce an experimental protocol that enables model comparisons based on equivalent compute, measured in accelerator hours. We pre-process an existing large, diverse, and high-quality dataset of books that surpasses existing academic benchmarks in quality, diversity, and document length. This work also provides two baseline models: a feed-forward model derived from the GPT-2 architecture and a recurrent model in the form of a novel LSTM with ten-fold throughput.
  arXiv  Detail & Related papers  (2023-09-20T10:31:17Z)
• CodeGen2: Lessons for Training LLMs on Programming and Natural Languages [116.74407069443895]
  We unify encoder and decoder-based models into a single prefix-LM. For learning methods, we explore the claim of a "free lunch" hypothesis. For data distributions, the effect of a mixture distribution and multi-epoch training of programming and natural languages on model performance is explored.
  arXiv  Detail & Related papers  (2023-05-03T17:55:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.