mSFT: Addressing Dataset Mixtures Overfitting Heterogeneously in Multi-task SFT

• URL: http://arxiv.org/abs/2603.21606v5
• Date: Fri, 27 Mar 2026 07:30:47 GMT
• Title: mSFT: Addressing Dataset Mixtures Overfitting Heterogeneously in Multi-task SFT
• Abstract summary: mSFT is an iterative, overfitting-aware search algorithm for multi-task data mixtures.<n>It consistently outperforms 4 baselines across 10 benchmarks and 6 base models.<n>At low compute budget, mSFT can improve performance while lowering training FLOPs.
• Score: 27.111441041682586
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current language model training commonly applies multi-task Supervised Fine-Tuning (SFT) using a homogeneous compute budget across all sub-datasets. This approach is fundamentally sub-optimal: heterogeneous learning dynamics cause faster-learning tasks to overfit early while slower ones remain under-fitted. To address this, we introduce mSFT, an iterative, overfitting-aware search algorithm for multi-task data mixtures. mSFT trains the model on an active mixture, identifies and excludes the earliest overfitting sub-dataset, and reverts to that specific optimal checkpoint before continuing. Extensive evaluations demonstrate that mSFT consistently outperforms 4 baselines across 10 benchmarks and 6 base models. Further analysis confirms mSFT maintains robust gains across diverse dataset sizes, task granularities, and is insensitive to its single new hyperparameter (compute budget). Notably, at low compute budget, mSFT can improve performance while lowering training FLOPs. Ultimately, mSFT establishes a practical overfitting-aware algorithm for multi-task SFT that maximizes the potential of models across diverse data mixtures.

Related papers

• Learning Multi-Indicator Weights for Data Selection: A Joint Task-Model Adaptation Framework with Efficient Proxies [50.39041754816285]
  We propose a framework for learning multi-validation weights that jointly adapts data selection to both the downstream task and the specific model.<n>Our approach achieves performance comparable to or exceeding full-dataset tuning while using only 30% of the training samples on GSM8K.
  arXiv  Detail & Related papers  (2026-05-10T17:30:16Z)
• FAAR: Efficient Frequency-Aware Multi-Task Fine-Tuning via Automatic Rank Selection [13.382659738606511]
  This paper introduces Frequency-Aware and Automatic Rank (FAAR) for efficient MTL fine-tuning.<n>FAAR reduces the number of parameters by up to 9 times compared to traditional MTL fine-tuning.
  arXiv  Detail & Related papers  (2026-03-20T18:24:06Z)
• Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning [49.04912820721943]
  Supervised fine-tuning (SFT) is computationally expensive and sometimes suffers from overfitting or bias amplification.<n>This work studies the online batch selection family that dynamically scores and filters samples during the training process.<n>We develop textbfUDS (Utility-Diversity Sampling), a framework for efficient online batch selection in SFT.
  arXiv  Detail & Related papers  (2025-10-19T15:32:01Z)
• Exploring Cross-Modal Flows for Few-Shot Learning [9.866094371902372]
  We propose a model-agnostic multi-step adjustment approach by learning a cross-modal velocity field: Flow Matching Alignment.<n>Results have demonstrated that FMA can consistently yield significant performance gains across various benchmarks and backbones.
  arXiv  Detail & Related papers  (2025-10-16T10:32:48Z)
• SPaRFT: Self-Paced Reinforcement Fine-Tuning for Large Language Models [51.74498855100541]
  Large language models (LLMs) have shown strong reasoning capabilities when fine-tuned with reinforcement learning (RL)<n>We propose textbfSPaRFT, a self-paced learning framework that enables efficient learning based on the capability of the model being trained.
  arXiv  Detail & Related papers  (2025-08-07T03:50:48Z)
• Why Reinforcement Fine-Tuning Enables MLLMs Preserve Prior Knowledge Better: A Data Perspective [98.45690529036848]
  Post-training algorithms such as Supervised Fine-Tuning (SFT) and Reinforcement Fine-Tuning (RFT) are widely used to adapt multimodal large language models to downstream tasks.<n>While effective at task adaptation, their impact on prior knowledge remains unclear.
  arXiv  Detail & Related papers  (2025-06-30T04:15:01Z)
• Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality [22.105116314320696]
  Supervised fine-tuning (SFT) is a critical step in aligning large language models with human instructions and values.<n>We trained a wide range of base models on a variety of datasets including code generation, mathematical reasoning, and general-domain tasks.<n>We then identified the dataset properties that matter most and examined the layer-wise modifications introduced by SFT.
  arXiv  Detail & Related papers  (2025-06-17T16:13:15Z)
• SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe [36.74756622715754]
  Large language models (LLMs) undergo instruction tuning to acquire instruction-following capabilities.<n>Efforts to improve instruction tuning often focus on higher-quality supervised fine-tuning datasets.<n>We propose SFTMix, a novel Mixup-based recipe that elevates LLM instruction tuning without relying on well-curated datasets.
  arXiv  Detail & Related papers  (2024-10-07T17:52:21Z)
• 60 Data Points are Sufficient to Fine-Tune LLMs for Question-Answering [50.12622877002846]
  Large language models (LLMs) encode extensive world knowledge through pre-training on massive datasets, which can be fine-tuned for the question-answering (QA) task.<n>We categorize supervised fine-tuning (SFT) data based on the extent of knowledge memorized by the pretrained LLMs.<n>Our experiments show that as few as 60 data points during the SFT stage can activate the knowledge encoded during pre-training, enabling LLMs to perform the QA task.
  arXiv  Detail & Related papers  (2024-09-24T07:38:38Z)
• Inducing Generalization across Languages and Tasks using Featurized Low-Rank Mixtures [44.04239492332651]
  Featurized Low-rank Mixtures (FLix) is a novel PEFT method designed for effective multitask multilingual adaptation. FLix associates each unique dataset feature, such as the dataset's language or task, with its own low-rank weight update parameters. Our experiments show that FLix leads to significant improvements over a variety of tasks for both supervised learning and zero-shot settings.
  arXiv  Detail & Related papers  (2024-02-27T23:12:45Z)
• Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule [55.08778142798106]
  We propose auxiliary tasks that exploit the alignment between the original and corrected sentences. We formulate each task as a sequence-to-sequence problem and perform multi-task training. We find that the order of datasets used for training and even individual instances within a dataset may have important effects on the final performance.
  arXiv  Detail & Related papers  (2023-11-20T14:50:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.