Models Can Model, But Can't Bind: Structured Grounding in Text-to-Optimization

• URL: http://arxiv.org/abs/2605.21751v1
• Date: Wed, 20 May 2026 21:25:41 GMT
• Title: Models Can Model, But Can't Bind: Structured Grounding in Text-to-Optimization
• Abstract summary: We find that accuracy collapses as instance data grows, even when the formulation itself is simple.<n>We address this via a simple inference-time approach, BIND, which externalizes numeric data to structured files so the model binds data rather than from prompt prompt.<n>We validate our hypothesis by finetuning a model exclusively on binding and show that it outperforms end-to-end SFT and RL across three structurally distinct optimization categories.
• Score: 54.749573452394664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-optimization requires two separable capabilities: modeling -- choosing the right optimization structure -- and binding -- grounding every coefficient, index, and parameter in the concrete problem data. We study this via Text2Opt-Bench, a scalable benchmark of solver-verified optimization problems spanning 12 categories, from textbook linear programs to stochastic and multi-objective formulations with up to thousands of variables. Across 10+ models, we find that accuracy collapses as instance data grows, even when the formulation itself is simple. We call this the effective binding limit. We address this via a simple inference-time approach, BIND, which externalizes numeric data to structured files so the model binds data programmatically rather than transcribing from the prompt. BIND improves GPT-5-Nano from 59.1% to 82.4% accuracy, matching pass@5 (82.0%) at lower token cost than pass@1, and GPT-5 from 86.2% to 95.8%. Furthermore, we validate our hypothesis by finetuning a model exclusively on binding and show that it outperforms end-to-end SFT and RL across three structurally distinct optimization categories, with a 1.5B binding specialist alone matching a 7B end-to-end baseline.

Related papers

• ToolLoop: Closed-Loop Tool-Use Data Synthesis via Decomposed Generation and Dynamic Self-Feedback [10.658946043856124]
  High-quality tool-use data is critical for training language models to interact effectively with external tools.<n>We propose ToolLoop, a closed-loop framework that decomposes synthesis into three progressive stages.<n>Dynamic self-feedback iteratively guides the model toward high-quality generation.
  arXiv  Detail & Related papers  (2026-09-08T17:26:35Z)
• CoMerge: Conflict-Driven Preference Optimization for Multi-Task Model Merging [59.836482609414816]
  We propose a conflict-driven preference optimization framework for model merging (CoMerge)<n>CoMerge enables the model to mitigate parameter-space conflicts while preserving task-specific capabilities.<n>Experiments show that CoMerge achieves an average normalized performance of 0.9968 on MergeBench.
  arXiv  Detail & Related papers  (2026-09-02T08:21:28Z)
• Bayesian Model Merging [17.887004278413915]
  Model merging aims to combine multiple task-specific expert models into a single model without joint retraining.<n>This paper introduces Bayesian Model Merging (BMM), a plug-and-play bi-level optimization framework.<n>BMM consistently outperforms all plug-and-play anchor baselines.
  arXiv  Detail & Related papers  (2026-05-13T00:36:47Z)
• VAREX: A Benchmark for Multi-Modal Structured Extraction from Documents [1.06378109904813]
  VAREX is a benchmark for evaluating structured data extraction from government forms.<n>The benchmark comprises 1,777 documents with 1,771 unique documents with validated ground truth through three-phase quality assurance.<n>Results reveal that below 4B parameters, compliance output -- not extraction capability -- is a dominant bottleneck.
  arXiv  Detail & Related papers  (2026-03-16T11:15:56Z)
• Robust LLM-based Column Type Annotation via Prompt Augmentation with LoRA Tuning [0.749377967268953]
  Column Type.<n>CTA is a fundamental step towards enabling schema alignment and semantic understanding.<n>Existing encoder-only language models achieve high accuracy when fine-tuned on labeled columns.<n>We present a parameter-efficient framework for CTA that trains models over prompt-augmented data.
  arXiv  Detail & Related papers  (2025-12-28T02:04:17Z)
• TinyLLM: Evaluation and Optimization of Small Language Models for Agentic Tasks on Edge Devices [0.0]
  This paper investigates the effectiveness of small language models (SLMs) for agentic tasks (function/tool/API calling)<n>We describe parameter-driven optimization strategies that include supervised fine-tuning (SFT), parameter-efficient fine-tuning (PEFT), reinforcement learning (RL), and hybrid methods.<n>Our results demonstrate clear accuracy differences across model scales where medium-sized models (1-3B parameters) significantly outperform ultra-compact models (1B parameters)<n>This study highlights the importance of hybrid optimization strategies that enable small language models to deliver accurate, efficient, and stable agentic AI on edge devices.
  arXiv  Detail & Related papers  (2025-11-27T06:09:54Z)
• Combatting Dimensional Collapse in LLM Pre-Training Data via Diversified File Selection [65.96556073745197]
  DiverSified File selection algorithm (DiSF) is proposed to select the most decorrelated text files in the feature space.<n>DiSF saves 98.5% of 590M training files in SlimPajama, outperforming the full-data pre-training within a 50B training budget.
  arXiv  Detail & Related papers  (2025-04-29T11:13:18Z)
• iTool: Reinforced Fine-Tuning with Dynamic Deficiency Calibration for Advanced Tool Use [56.31110409360567]
  Augmenting large language models with external tools is a promising approach to enhance their capabilities.<n>We show that training gains significantly decay as synthetic data increases.<n>We propose an iterative reinforced fine-tuning strategy designed to alleviate this limitation.
  arXiv  Detail & Related papers  (2025-01-15T04:52:34Z)
• Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Prompting [107.4034346788744]
  Existing vehicle trajectory prediction models struggle with generalizability, prediction uncertainties, and handling complex interactions.<n>We propose Perceiver with Register queries (PerReg+), a novel trajectory prediction framework that introduces: (1) Dual-Level Representation Learning via Self-Distillation (SD) and Masked Reconstruction (MR), capturing global context and fine-grained details; (2) Enhanced Multimodality using register-based queries and pretraining, eliminating the need for clustering and suppression; and (3) Adaptive Prompt Tuning during fine-tuning, freezing the main architecture and optimizing a small number of prompts for efficient adaptation.
  arXiv  Detail & Related papers  (2025-01-08T20:11:09Z)
• Crafting Efficient Fine-Tuning Strategies for Large Language Models [2.633490094119608]
  Fine-tuning large language models (LLMs) with as few as 200 samples can improve model accuracy from 70% to 88% in a product attribute extraction task. A bayesian hyperparameter optimization method, which evaluates models at 20% of total training time, correlates strongly with final model performance. This approach led to a 2% improvement in accuracy over baseline models when evaluated on an independent test set.
  arXiv  Detail & Related papers  (2024-07-18T21:36:00Z)
• Self-Play Preference Optimization for Language Model Alignment [75.83359213697854]
  Recent advancements suggest that directly working with preference probabilities can yield a more accurate reflection of human preferences. We propose a self-play-based method for language model alignment, which treats the problem as a constant-sum two-player game. Our approach, dubbed Self-Play Preference Optimization (SPPO), utilizes iterative policy updates to provably approximate the Nash equilibrium.
  arXiv  Detail & Related papers  (2024-05-01T17:59:20Z)
• Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning [126.84770886628833]
  Existing finetuning methods either tune all parameters of the pretrained model (full finetuning) or only tune the last linear layer (linear probing) We propose a new parameter-efficient finetuning method termed as SSF, representing that researchers only need to Scale and Shift the deep Features extracted by a pre-trained model to catch up with the performance full finetuning.
  arXiv  Detail & Related papers  (2022-10-17T08:14:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.