LeGo-Code: Can Modular Curriculum Learning Advance Complex Code Generation? Insights from Text-to-SQL

• URL: http://arxiv.org/abs/2604.18254v1
• Date: Mon, 20 Apr 2026 13:30:04 GMT
• Title: LeGo-Code: Can Modular Curriculum Learning Advance Complex Code Generation? Insights from Text-to-SQL
• Abstract summary: We investigate whether curriculum learning can improve the performance of code-based adapters on Text-to- relational tasks.<n>Our experiments show that naive curriculum, simply ordering training samples by complexity in a single epoch, fails to surpass standard fine-tuning due to catastrophic forgetting.<n>By sequentially training tier-specific adapters on incremental complexity levels, we create a scaffolded learning environment that improves performance on complex queries.
• Score: 7.040712790568155
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recently, code-oriented large language models (LLMs) have demonstrated strong capabilities in translating natural language into executable code. Text-to-SQL is a significant application of this ability, enabling non-technical users to interact with relational databases using natural language. However, state-of-the-art models continue to struggle with highly complex logic, particularly deeply nested statements involving multiple joins and conditions, as well as with real-world database schemas that are noisy or poorly structured. In this paper, we investigate whether curriculum learning can improve the performance of code-based LLMs on Text-to-SQL tasks. Employing benchmarks including Spider and BIRD, we fine-tune models under different curriculum strategies. Our experiments show that naive curriculum, simply ordering training samples by complexity in a single epoch, fails to surpass standard fine-tuning due to catastrophic forgetting. To overcome this, we propose a Modular Adapter Composition (MAC) strategy. By sequentially training tier-specific adapters on incremental complexity levels (Easy to Extra-Hard), we create a scaffolded learning environment that improves performance on complex queries. Our approach not only produces measurable performance gains on the Spider and BIRD benchmarks but also provides a flexible, "Lego-like" architecture, allowing models to be composed and deployed based on specific schema difficulty requirements. These findings demonstrate that structured, modular learning is a superior alternative to monolithic fine-tuning for mastering the syntax and logic of complex code generation.

Related papers

• RealBench: A Repo-Level Code Generation Benchmark Aligned with Real-World Software Development Practices [54.956760584923295]
  Researchers have made substantial progress using Large Language Models (LLMs) for code generation.<n>However, developers typically write code based on structured designs or specifications rather than raw natural language descriptions.<n>This gap between existing benchmarks and real industry development practices means that current benchmark scores may not accurately reflect how much code generation can help automate development tasks.
  arXiv  Detail & Related papers  (2026-04-24T15:35:54Z)
• Learning to Solve Complex Problems via Dataset Decomposition [53.1641602054716]
  This research explores a reverse curriculum generation approach that decomposes complex datasets into simpler, more learnable components.<n>We propose a teacher-student framework where the teacher is equipped with the ability to reason step-by-step, which is used to generate easier versions of examples.
  arXiv  Detail & Related papers  (2026-02-23T19:25:40Z)
• IESR:Efficient MCTS-Based Modular Reasoning for Text-to-SQL with Large Language Models [10.758655501692793]
  We propose a framework named IESR(Information Enhanced Structured Reasoning) for lightweight large language models.<n>We show that IESR achieves state-of-the-art performance on the complex reasoning benchmark LogicCat and the Archer dataset.<n>Our analysis reveals that current coder models exhibit notable biases and deficiencies in physical knowledge, mathematical computation, and common-sense reasoning.
  arXiv  Detail & Related papers  (2026-02-05T07:10:45Z)
• EvolSQL: Structure-Aware Evolution for Scalable Text-to-SQL Data Synthesis [25.689983072200047]
  Evol is a structure-aware data synthesis framework that evolves queries into richer and more semantically diverse forms.<n>A 7B model outperforms one trained on the much larger Syn dataset using only 1/18 of the data.
  arXiv  Detail & Related papers  (2026-01-08T12:19:50Z)
• High-Fidelity And Complex Test Data Generation For Google SQL Code Generation Services [0.8177323519057186]
  The demand for high-fidelity test data is paramount in industrial settings where access to production data is largely restricted.<n>Traditional data generation methods often fall short, struggling with low-fidelity and the ability to model complex data structures and semantic relationships.<n>We demonstrate that by leveraging Large Language Models (LLMs) and incorporating strategic pre- and post-processing steps, we can generate syntactically correct and semantically relevant high-fidelity test data.
  arXiv  Detail & Related papers  (2025-04-24T02:27:17Z)
• EpiCoder: Encompassing Diversity and Complexity in Code Generation [66.43738008739555]
  Existing methods for code generation use code snippets as seed data.<n>We introduce a novel feature tree-based synthesis framework, which revolves around hierarchical code features.<n>Our framework provides precise control over the complexity of the generated code, enabling functionalities that range from function-level operations to multi-file scenarios.
  arXiv  Detail & Related papers  (2025-01-08T18:58:15Z)
• Structure Guided Large Language Model for SQL Generation [14.079764882536077]
  We propose a novel structure-aware text-to- query and framework(SGU)<n>SGU-aware text-to- query and framework(SGU) consistently outperforms state-of-the-art text-to-models.
  arXiv  Detail & Related papers  (2024-02-19T09:07:59Z)
• Semantic Parsing for Complex Data Retrieval: Targeting Query Plans vs. SQL for No-Code Access to Relational Databases [2.933060994339853]
  We investigate the potential of an alternative query language with simpler syntax and modular specification of complex queries. The proposed alternative query language is called Query Plan Language (QPL) We present ways to address the challenge of complex queries in an iterative, user-controlled manner.
  arXiv  Detail & Related papers  (2023-12-22T16:16:15Z)
• Can Large Language Models Understand Real-World Complex Instructions? [54.86632921036983]
  Large language models (LLMs) can understand human instructions, but struggle with complex instructions. Existing benchmarks are insufficient to assess LLMs' ability to understand complex instructions. We propose CELLO, a benchmark for evaluating LLMs' ability to follow complex instructions systematically.
  arXiv  Detail & Related papers  (2023-09-17T04:18:39Z)
• When Do Program-of-Thoughts Work for Reasoning? [51.2699797837818]
  We propose complexity-impacted reasoning score (CIRS) to measure correlation between code and reasoning abilities. Specifically, we use the abstract syntax tree to encode the structural information and calculate logical complexity. Code will be integrated into the EasyInstruct framework at https://github.com/zjunlp/EasyInstruct.
  arXiv  Detail & Related papers  (2023-08-29T17:22:39Z)
• SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended) [53.95151604061761]
  This paper introduces the framework for enhancing Text-to- filtering using large language models (LLMs) With few-shot prompting, we explore the effectiveness of consistency decoding with execution-based error analyses. With instruction fine-tuning, we delve deep in understanding the critical paradigms that influence the performance of tuned LLMs.
  arXiv  Detail & Related papers  (2023-05-26T21:39:05Z)
• Learning Label Modular Prompts for Text Classification in the Wild [56.66187728534808]
  We propose text classification in-the-wild, which introduces different non-stationary training/testing stages. Decomposing a complex task into modular components can enable robust generalisation under such non-stationary environment. We propose MODULARPROMPT, a label-modular prompt tuning framework for text classification tasks.
  arXiv  Detail & Related papers  (2022-11-30T16:26:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.