Distilling Foundation Models for Agentic What-If Reasoning:Cost, Latency, and Governance in a Hybrid LLM+SLM Architecture

• URL: http://arxiv.org/abs/2609.16091v1
• Date: Mon, 14 Sep 2026 11:26:21 GMT
• Title: Distilling Foundation Models for Agentic What-If Reasoning:Cost, Latency, and Governance in a Hybrid LLM+SLM Architecture
• Abstract summary: Tabular foundation models deliver strong zero-training predictive performance via in-context learning, but their high inference latency makes them impractical as hot-path decision backends in interactive agentic loops.<n>We distill a TabPFN teacher into a compact feed-forward student across a business-decision simulation on UCI Adult and five OpenML benchmarks.<n>The student retains 95.4-100.5% accuracy and 96.8-100.0% AUC, with the lowest accuracy retention on credit-g at 95.4%.
• Score: 4.205733518246009
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tabular foundation models deliver strong zero-training predictive performance via in-context learning, but their high inference latency makes them impractical as hot-path decision backends in interactive agentic loops. We distill a TabPFN teacher into a compact feed-forward student across a business-decision simulation on UCI Adult and five OpenML benchmarks: the classification head compresses 53.2M parameters to 8,546 (6,220x); the deployed two-head loan pipeline compresses 111.4M parameters to 17,059 (6,532x). The student retains 95.4-100.5% accuracy and 96.8-100.0% AUC, with the lowest accuracy retention on credit-g at 95.4%; an alpha = 0 hard-label control shows that the teacher's soft targets provide a 2.1-7.0 AUC point gain.

Related papers

• ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search [54.43696074323815]
  ZGCM-1 is a fully open 7B foundation model trained from scratch with extreme data, system, and algorithmic efficiency.<n>We establish an AI-native R&D workflow where agent swarms autonomously manage cluster operations.<n>ZGCM-1-7B is competitive across 7B model family on general benchmarks.
  arXiv  Detail & Related papers  (2026-09-11T17:18:04Z)
• Distill Globally, Adapt Locally: Reasoning Distillation and Product-Type Test-Time Training for Scalable Trade-Up Recommendation [4.571228446381617]
  Trade-up recommendation identifies higher-quality alternatives that preserve a customer's purchase intent while offering upgraded benefits.<n>Large language models (LLMs) can reason about such distinctions, but applying them directly to hundreds of millions of product pairs is operationally impractical.<n>We introduce a two-level framework that distills LLM reasoning into an efficient non-generative student and adapts its decision boundary to product-type-specific trade-up criteria.
  arXiv  Detail & Related papers  (2026-09-04T17:08:50Z)
• Annotations as Rollouts: Efficient and Scalable Reinforcement Learning for Video MLLMs [104.11820606942855]
  Video-ORA-9B decodes in 130 ms instead of 4,780 ms without chain-of-thought.<n>OraRL scales with model size and data, surpassing its backbone from 0.8B to 9B and GRPO up to 100k prompts.
  arXiv  Detail & Related papers  (2026-08-20T18:28:19Z)
• Models for minimalist RAG: B1ade 335M Embedding and 1B Parameter Small Language Models [2.1665689529884697]
  We present B1ade, an efficient RAG architecture comprising two purpose-built components: a compact embedding model and a purpose-built SLM.<n>B1ade-embed, a 335M parameter retrieval model constructed via parameter-free fusion of five pretrained encoders achieves top MTEB scores among sub-500M models with zero additional training.<n>B1ade-1B cites retrieved passages in 42.4% of responses, exceeding the attribution rate of its training distribution by 5.5 percentage points.
  arXiv  Detail & Related papers  (2026-07-29T22:41:25Z)
• Multi-Teacher Contrastive Distillation for Edge-Efficient Pathology Foundation Models [1.9573380763700714]
  We propose MuCoDi, a pretraining framework that distills frozen tile embeddings from multiple PFMs into compact edge-oriented encoders.<n>We pretrain students on 14.3M TCGA tiles from only 11.8K WSIs and evaluate frozen encoders on 23 clinically curated downstream classification tasks.
  arXiv  Detail & Related papers  (2026-07-06T18:14:37Z)
• Knowledge Distillation from Large Reasoning Models to Compact Student Models: A Case Study on the John O Bryan Mathematics Competition [0.0]
  We build a Chain-of-Thought training corpus through a dual-agent framework.<n>The dataset is used to fine-tune the student model with Low-Rank Adaptation (LoRA) on Apple Silicon hardware.
  arXiv  Detail & Related papers  (2026-06-30T02:34:45Z)
• FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization [84.58281577727566]
  We present Future-KL Influenced Policy Optimization (FIPO), a reinforcement learning algorithm designed to overcome reasoning bottlenecks in large language models.<n>FIPO addresses this by incorporating discounted future-KL divergence into the policy update, creating a dense advantage formulation that re-weights tokens based on their influence on subsequent trajectory behavior.<n> evaluated on Qwen2.5-32B, FIPO extends the average chain-of-thought length from roughly 4,000 to over 10,000 tokens and increases AIME 2024 Pass@1 accuracy from 50.0% to a peak of 58.0%.
  arXiv  Detail & Related papers  (2026-03-20T10:24:50Z)
• Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters [169.7981969517903]
  Step 3.5 Flash bridges frontier-level agentic intelligence and computational efficiency.<n>We focus on what matters most when building agents: sharp reasoning and fast, reliable execution.
  arXiv  Detail & Related papers  (2026-02-11T07:53:51Z)
• SmolTulu: Higher Learning Rate to Batch Size Ratios Can Lead to Better Reasoning in SLMs [0.0]
  We present SmolTulu, an instruction-tuned language model that adapts AllenAI's Tulu 3 post-training pipeline to enhance Huggingface's SmolLM2-1.7B base model.<n>Our findings reveal a clear split: reasoning tasks like ARC and GSM8K benefit from higher learning rate to batch size ratios, while pattern recognition tasks such as HellaSwag and IFEval show optimal performance with lower ratios.
  arXiv  Detail & Related papers  (2024-12-11T12:41:36Z)
• Building Math Agents with Multi-Turn Iterative Preference Learning [56.71330214021884]
  This paper studies the complementary direct preference learning approach to further improve model performance.<n>Existing direct preference learning algorithms are originally designed for the single-turn chat task.<n>We introduce a multi-turn direct preference learning framework, tailored for this context.
  arXiv  Detail & Related papers  (2024-09-04T02:41:04Z)
• Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems [63.713297451300086]
  We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9.3B. Their subsequent distillation into smaller models ranging from 17M-170M parameters, and their application to the Natural Language Understanding (NLU) component of a virtual assistant system.
  arXiv  Detail & Related papers  (2022-06-15T20:44:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.