STAIF: A Stage-wise Optimization for Complex Instruction Following

• URL: http://arxiv.org/abs/2607.22649v1
• Date: Fri, 26 Jun 2026 06:14:07 GMT
• Title: STAIF: A Stage-wise Optimization for Complex Instruction Following
• Abstract summary: STAIF is a stage-wise optimization framework that decouples the alignment of subjective (soft) constraints from the optimization of objectively verifiable (hard) constraints.<n>To support this method, we construct STAINSTRUCT, a high-quality bilingual (English, Chinese) dataset of approximately 31,000 complex multi-constraint instructions.
• Score: 50.48101504556211
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Following complex instructions with multiple explicit constraints remains a fundamental challenge for large language models (LLMs). Existing alignment methods, such as DPO, optimize holistic reward signals that often underemphasize strict satisfaction of individual constraints, particularly under out-of-distribution or multi-constraint settings. In this paper, we propose STAIF, a stage-wise optimization framework that decouples the alignment of subjective (soft) constraints from the optimization of objectively verifiable (hard) constraints. Stage 1 applies preference optimization with multiple negative samples to sharpen sensitivity to soft constraints, while Stage 2 applies Reinforcement Learning with Verifiable Rewards (RLVR) to enforce strict compliance with hard constraints. To support this method, we construct STAINSTRUCT, a high-quality bilingual (English, Chinese) dataset of approximately 31,000 complex multi-constraint instructions. Extensive analyses validate the design of STAIF and show state-of-the-art performance on representative benchmarks against strong baselines, as well as genuine generalization.

Related papers

• Multi-Objective Kinodynamic Motion Planning with Asymptotic Pareto Optimality [8.518351216651062]
  We address the challenge of multi-objective motion planning for systems under kinodynamic constraints.<n>We propose a unified algorithmic framework built upon the Stable Sparse-RRT (SST) algorithm.
  arXiv  Detail & Related papers  (2026-07-16T23:32:59Z)
• Feasibility Restoration under Conflicting STL Specifications with Pareto-Optimal Refinement [18.383508411056944]
  Signal Temporal Logic (STL) is expressive language that specifies expressive requirements in robotics.<n>STL specifications may become conflicting in real-world applications, where safety rules, traffic regulations, task and objectives can be cannot be satisfied together.<n>We propose a unified two-stage framework that first restores feasibility via minimal relaxation, then refine the feasible solution by formulating it as a value-aware multi-objective optimization problem.
  arXiv  Detail & Related papers  (2026-03-06T23:41:01Z)
• OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL [63.388513841293616]
  Existing forgery detection methods fail to handle the interleaved text, images, and videos prevalent in real-world misinformation.<n>To bridge this gap, this paper targets to develop a unified framework for omnibus vision-language forgery detection and grounding.<n>We propose textbf OmniVL-Guard, a balanced reinforcement learning framework for omnibus vision-language forgery detection and grounding.
  arXiv  Detail & Related papers  (2026-02-11T09:41:36Z)
• Multimodal Large Language Models with Adaptive Preference Optimization for Sequential Recommendation [60.33386541343322]
  We propose a Multimodal Large Language Models framework that integrates Hardness-aware and Noise-regularized preference optimization for Recommendation (HaNoRec)<n>Specifically, HaNoRec dynamically adjusts optimization weights based on both the estimated hardness of each training sample and the policy model's real-time responsiveness.
  arXiv  Detail & Related papers  (2025-11-24T04:10:46Z)
• Light-IF: Endowing LLMs with Generalizable Reasoning via Preview and Self-Checking for Complex Instruction Following [10.119219532863767]
  lazy reasoning during the thinking stage is the primary factor contributing to poor instruction adherence.<n>We propose a comprehensive framework designed to enable rigorous reasoning processes involving preview and self-checking.<n>Our Light-IF-32B model surpasses both larger open-source models such as DeepSeek-R1 and closed-source models like Doubao-1.6.
  arXiv  Detail & Related papers  (2025-08-05T07:42:00Z)
• RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Data [47.19854998380304]
  RECAST is an efficient framework for synthesizing datasets with far more constraints than those in existing benchmarks.<n>We construct RECAST-30K, a large-scale, high-quality dataset comprising 30k instances spanning 19 constraint types.<n> Experimental results demonstrate that models finetuned on RECAST-30K substantially improve in following complex instructions.
  arXiv  Detail & Related papers  (2025-05-25T08:31:08Z)
• RBF++: Quantifying and Optimizing Reasoning Boundaries across Measurable and Unmeasurable Capabilities for Chain-of-Thought Reasoning [60.84707424369494]
  Chain-of-Thought (CoT) reasoning has proven effective in enhancing large language models (LLMs) on complex tasks.<n>We introduce the Reasoning Boundary Framework++ (RBF++), a framework for evaluating and optimizing measurable boundaries of CoT capability.
  arXiv  Detail & Related papers  (2025-05-19T16:25:55Z)
• MuSC: Improving Complex Instruction Following with Multi-granularity Self-Contrastive Training [36.483136685734735]
  We propose a Multi-granularity Self-Contrastive Training (MuSC) framework to improve the complex instruction alignment without relying on a stronger model.<n>Our method is evaluated on open-sourced models, and experiment results show our method achieves significant improvement on both complex and general instruction-following benchmarks.
  arXiv  Detail & Related papers  (2025-02-17T08:12:49Z)
• Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Models [39.114513139453756]
  It is crucial for large language models (LLMs) to follow instructions that involve multiple constraints.<n>We design a pipeline to construct datasets with high-quality outputs automatically.<n>To fully utilize the positive and negative samples generated during the data construction process, we choose Direct Preference Optimization (DPO) as the training method.<n>We experimentally evaluate the effectiveness of our methods in improving LLMs' soft constraint following ability.
  arXiv  Detail & Related papers  (2025-01-09T03:34:07Z)
• Multi-Attribute Constraint Satisfaction via Language Model Rewriting [67.5778646504987]
  Multi-Attribute Constraint Satisfaction (MACS) is a method capable of finetuning language models to satisfy user-specified constraints on multiple external real-value attributes.<n>Our work opens new avenues for generalized and real-value multi-attribute control, with implications for diverse applications spanning NLP and bioinformatics.
  arXiv  Detail & Related papers  (2024-12-26T12:36:39Z)
• A Systematic Examination of Preference Learning through the Lens of Instruction-Following [83.71180850955679]
  We use a novel synthetic data generation pipeline to generate 48,000 instruction unique-following prompts.<n>With our synthetic prompts, we use two preference dataset curation methods - rejection sampling (RS) and Monte Carlo Tree Search (MCTS)<n>Experiments reveal that shared prefixes in preference pairs, as generated by MCTS, provide marginal but consistent improvements.<n>High-contrast preference pairs generally outperform low-contrast pairs; however, combining both often yields the best performance.
  arXiv  Detail & Related papers  (2024-12-18T15:38:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.