Canonical Intermediate Representation for LLM-based optimization problem formulation and code generation

• URL: http://arxiv.org/abs/2602.02029v1
• Date: Mon, 02 Feb 2026 12:26:27 GMT
• Title: Canonical Intermediate Representation for LLM-based optimization problem formulation and code generation
• Abstract summary: We introduce the Canonical Intermediate Representation (CIR), a schema that LLMs explicitly generate between problem descriptions and optimization models.<n>CIR encodes the semantics of operational rules through constraint archetypes and candidate modeling paradigms.<n>We develop the rule-to-constraint framework, a multi-agent pipeline that parses problem texts, synthesizes CIR implementations by retrieving domain knowledge, and instantiates optimization models.
• Score: 22.998405337017928
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatically formulating optimization models from natural language descriptions is a growing focus in operations research, yet current LLM-based approaches struggle with the composite constraints and appropriate modeling paradigms required by complex operational rules. To address this, we introduce the Canonical Intermediate Representation (CIR): a schema that LLMs explicitly generate between problem descriptions and optimization models. CIR encodes the semantics of operational rules through constraint archetypes and candidate modeling paradigms, thereby decoupling rule logic from its mathematical instantiation. Upon a newly generated CIR knowledge base, we develop the rule-to-constraint (R2C) framework, a multi-agent pipeline that parses problem texts, synthesizes CIR implementations by retrieving domain knowledge, and instantiates optimization models. To systematically evaluate rule-to-constraint reasoning, we test R2C on our newly constructed benchmark featuring rich operational rules, and benchmarks from prior work. Extensive experiments show that R2C achieves state-of-the-art accuracy on the proposed benchmark (47.2% Accuracy Rate). On established benchmarks from the literature, R2C delivers highly competitive results, approaching the performance of proprietary models (e.g., GPT-5). Moreover, with a reflection mechanism, R2C achieves further gains and sets new best-reported results on some benchmarks.

Related papers

• IR2Solve: Structured Intermediate Representations for Cost-Efficient Optimization Autoformulation [15.520617657780578]
  Large language models (LLMs) can translate natural-language optimization problems into solver-ready formulations, but direct code generation is brittle.<n>We present IR2, an intermediate-representation-first autoformulation pipeline that uses a single semantic call to produce a schema-constrained ModelIR.<n>A concrete scalar-constraint convention represents finite per-index constraint families as individual entries, reducing free-index and implicit-quantification errors.
  arXiv  Detail & Related papers  (2026-07-31T13:36:26Z)
• StarOR: Synergizing Tree Search and Test-Time Reinforcement Learning for Optimization Modeling [13.71352763893512]
  We propose StarOR, a search-and-adaptation framework that couples MCTS with Test-Time Reinforcement Learning for optimization modeling.<n>StarOR decomposes the modeling process into four stages and updates a transient LoRA adapter via GRPO at each non-terminal node.<n> Experiments across five optimization benchmarks show that StarOR achieves state-of-the-art performance even with a 4B backbone.
  arXiv  Detail & Related papers  (2026-06-13T08:46:13Z)
• MIRROR: A Multi-Agent Framework with Iterative Adaptive Revision and Hierarchical Retrieval for Optimization Modeling in Operations Research [15.28095645151852]
  MIRROR is a fine-tuning-free, end-to-end multi-agent framework for operations research.<n>It translates natural language optimization problems into mathematical models and solver code.<n>Experiments show that MIRROR outperforms existing methods on standard Operations Research benchmarks.
  arXiv  Detail & Related papers  (2026-02-03T09:46:56Z)
• Deep Unfolding: Recent Developments, Theory, and Design Guidelines [99.63555420898554]
  This article provides a tutorial-style overview of deep unfolding, a framework that transforms optimization algorithms into structured, trainable ML architectures.<n>We review the foundations of optimization for inference and for learning, introduce four representative design paradigms for deep unfolding, and discuss the distinctive training schemes that arise from their iterative nature.
  arXiv  Detail & Related papers  (2025-12-03T13:16:35Z)
• ConstraintLLM: A Neuro-Symbolic Framework for Industrial-Level Constraint Programming [9.69748612176497]
  Constraint programming (CP) is a crucial technology for solving real-world constraint optimization problems (COPs)<n>Using large language models (LLMs) to generate formal modeling automatically for COPs is becoming a promising approach, which aims to build neuro-symbolic AI with the help of symbolic solvers.<n>We introduce ConstraintLLM, the first LLM specifically designed for CP modeling, which is trained on an open-source LLM with multi-instruction supervised fine-tuning.<n>We construct and release IndusCP, the first industrial-level benchmark for CP modeling, which contains 140 challenging tasks from various domains
  arXiv  Detail & Related papers  (2025-10-07T10:43:39Z)
• CALM Before the STORM: Unlocking Native Reasoning for Optimization Modeling [60.55856973678002]
  Large Reasoning Models (LRMs) have demonstrated strong capabilities in complex multi-step reasoning.<n>Existing domain adaptation methods, originally designed for earlier instruction-tuned models, often fail to exploit the advanced reasoning patterns of modern LRMs.<n>We propose textbfCALM, a framework that progressively refines LRMs within their native reasoning modes for optimization modeling tasks.
  arXiv  Detail & Related papers  (2025-10-05T13:38:31Z)
• SCoRE: Streamlined Corpus-based Relation Extraction using Multi-Label Contrastive Learning and Bayesian kNN [0.2812395851874055]
  We introduce SCoRE, a modular and cost-effective sentence-level relation extraction system.<n>SCoRE enables easy PLM switching, requires no finetuning, and adapts smoothly to diverse corpora and KGs.<n>We show that SCoRE matches or surpasses state-of-the-art methods while significantly reducing energy consumption.
  arXiv  Detail & Related papers  (2025-07-09T14:33:07Z)
• Walk Before You Run! Concise LLM Reasoning via Reinforcement Learning [10.255235456427037]
  We propose a simple yet effective two-stage reinforcement learning framework for achieving concise reasoning in Large Language Models (LLMs)<n>The first stage, using more training steps, aims to incentivize the model's reasoning capabilities via Group Relative Policy Optimization.<n>The second stage, using fewer training steps, explicitly enforces conciseness and improves efficiency via Length-aware Group Relative Policy Optimization.
  arXiv  Detail & Related papers  (2025-05-27T13:29:51Z)
• T2I-Eval-R1: Reinforcement Learning-Driven Reasoning for Interpretable Text-to-Image Evaluation [60.620408007636016]
  We propose T2I-Eval-R1, a novel reinforcement learning framework that trains open-source MLLMs using only coarse-grained quality scores.<n>Our approach integrates Group Relative Policy Optimization into the instruction-tuning process, enabling models to generate both scalar scores and interpretable reasoning chains.
  arXiv  Detail & Related papers  (2025-05-23T13:44:59Z)
• Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Models [83.8639566087953]
  We propose a direct retrieval-augmented optimization framework, named DRO, that enables end-to-end training of two key components.<n>DRO alternates between two phases: (i) document permutation estimation and (ii) re-weighted, progressively improving RAG components.<n>Our theoretical analysis reveals that DRO is analogous to policy-gradient methods in reinforcement learning.
  arXiv  Detail & Related papers  (2025-05-05T23:54:53Z)
• MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Search [61.11836311160951]
  We introduce MCTS-RAG, a novel approach that enhances the reasoning capabilities of small language models on knowledge-intensive tasks.<n>Unlike standard RAG methods, which typically retrieve information independently from reasoning, MCTS-RAG combines structured reasoning with adaptive retrieval.<n>This integrated approach enhances decision-making, reduces hallucinations, and ensures improved factual accuracy and response consistency.
  arXiv  Detail & Related papers  (2025-03-26T17:46:08Z)
• Reward-Guided Speculative Decoding for Efficient LLM Reasoning [80.55186052123196]
  We introduce Reward-Guided Speculative Decoding (RSD), a novel framework aimed at improving the efficiency of inference in large language models (LLMs)<n>RSD incorporates a controlled bias to prioritize high-reward outputs, in contrast to existing speculative decoding methods that enforce strict unbiasedness.<n>RSD delivers significant efficiency gains against decoding with the target model only, while achieving significant better accuracy than parallel decoding method on average.
  arXiv  Detail & Related papers  (2025-01-31T17:19:57Z)
• CTC-Assisted LLM-Based Contextual ASR [40.6542391788212]
  We propose a CTC-Assisted LLM-Based Contextual ASR model with an efficient filtering algorithm. Our model attains WER/B-WER of 1.27%/3.67% and 2.72%/8.02% on the Librispeech test-clean and test-other sets targeting on recognizing rare long-tail words.
  arXiv  Detail & Related papers  (2024-11-10T11:47:50Z)
• In-Context Symbolic Regression: Leveraging Large Language Models for Function Discovery [5.2387832710686695]
  In this work, we introduce the first comprehensive framework that utilizes Large Language Models (LLMs) for the task of Symbolic Regression. We propose In-Context Symbolic Regression (ICSR), an SR method which iteratively refines a functional form with an external LLM and determines its coefficients with an external LLM. Our findings reveal that LLMs are able to successfully find symbolic equations that fit the given data, matching or outperforming the overall performance of the best SR baselines on four popular benchmarks.
  arXiv  Detail & Related papers  (2024-04-29T20:19:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.