Diversity in Large Language Models under Supervised Fine-Tuning

• URL: http://arxiv.org/abs/2605.00195v1
• Date: Thu, 30 Apr 2026 20:20:59 GMT
• Title: Diversity in Large Language Models under Supervised Fine-Tuning
• Abstract summary: Supervised Fine-Tuning (SFT) is essential for aligning Large Language Models (LLMs) with user intent, yet it is believed to suppress generative diversity.<n>In this study, we attribute the decline to two primary drivers: the forgetting of preexisting knowledge and low-frequency patterns within finetuning datasets.<n>Our evaluation confirms at scale that generation narrows after SFT and strengthens the hypothesis explaining this effect.
• Score: 15.119045051735633
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Supervised Fine-Tuning (SFT) is essential for aligning Large Language Models (LLMs) with user intent, yet it is believed to suppress generative diversity. Although this reduction is frequently referenced, formal empirical testing of the phenomenon remains limited. The expressiveness of LLMs by itself was addressed by multiple prior methods. Their varying perspectives suggest that deeper analysis could yield further improvements. In this study, we attribute the decline to two primary drivers: the neglect of low-frequency patterns within fine-tuning datasets and the forgetting of preexisting knowledge. Motivated by our theoretical analysis, we develop Tempered Focal (TOFU) loss, a novel objective that addresses both stated challenges simultaneously. Our extensive evaluation confirms at scale that generation breadth narrows after SFT and strengthens the hypothesis explaining this effect. Across multiple models and benchmarks, we demonstrate that TOFU enhances output diversity while preserving high response quality, offering a principled approach to SFT.

Related papers

• Unified Hallucination Fuzzing for Multimodal Large Language Models [44.4785399881315]
  Hallucination remains a persistent challenge for Multimodal Large Language Models (MLLMs)<n>Existing evaluations, predominantly based on static benchmarks, suffer from narrow taxonomical coverage and rapid performance saturation.<n>We present a systematic evaluation framework integrating a comprehensive benchmark with self-evolving stress testing.
  arXiv  Detail & Related papers  (2026-07-15T17:57:37Z)
• On the Hidden Costs of Counterfactual Knowledge Training in LLM Unlearning [65.58724910851039]
  Counterfactual tuning (CFT) has emerged as a promising paradigm for Large Language Model (LLM) unlearning by training models.<n>We find that this paradigm still underperforms other paradigms in some aspects.<n>We introduce RWKU+, an extended benchmark equipped with novel trade-off metrics and gradient-level diagnostic tools.
  arXiv  Detail & Related papers  (2026-05-26T14:34:14Z)
• AnE: Pushing the Reasoning Frontier of Multimodal LLMs via Anchor Evolution [61.593935260052795]
  Post-training viaSupervised Fine-Tuning (SFT) and Reinforcement Learning (RL) is crucial for enhancing reasoning in Multimodal Large Language Models (MLLMs)<n>Existing paradigms often reach a performance bottleneck due to the limitations of static data.<n>We propose Anchor Evolution (AnE), a new paradigm that integrates truth-anchored data curation and model evolution.
  arXiv  Detail & Related papers  (2026-05-25T08:26:34Z)
• Perception Without Engagement: Dissecting the Causal Discovery Deficit in LMMs [68.58207076756237]
  We introduce ProCauEval, a perturbation-based evaluation protocol that shifts from outcome assessment to mechanism diagnosis.<n>We find that models faithfully perceive video content yet systematically underexploit it during causal reasoning.<n>We propose Anti-Distillation Policy Optimization (ADPO), a reinforcement learning framework built on negative teacher alignment.
  arXiv  Detail & Related papers  (2026-05-10T08:48:58Z)
• Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models [53.15468578562038]
  We propose an Uncertainty-aware Exploratory Direct Preference Optimization (UE-DPO) method for Multimodal Large Language Models (MLLMs)<n>We first quantify the uncertainty from the model's failure to ground token predictions in the given image.<n>We then encourage more learning pressure on visually deficient tokens in preferred samples, and alleviate the over-penalization of beneficial knowledge in dispreferred samples.
  arXiv  Detail & Related papers  (2026-05-06T13:08:12Z)
• MSA-Thinker: Discrimination-Calibration Reasoning with Hint-Guided Reinforcement Learning for Multimodal Sentiment Analysis [5.1150258716324055]
  Multimodal sentiment analysis aims to understand human emotions by integrating textual, auditory, and visual modalities.<n>Existing methods incorporating Chain-of-Thought (CoT) reasoning are hindered by high annotation costs.<n>We propose a novel training framework that integrates structured Discrimination-Calibration (DC) reasoning with Hint-based Reinforcement Learning.
  arXiv  Detail & Related papers  (2026-03-10T12:48:41Z)
• Scaling Test-Time Robustness of Vision-Language Models via Self-Critical Inference Framework [20.533408486872702]
  Large Language Models (LLMs) have driven rapid progress in multi-modal learning, particularly in the development of Large Vision-Language Models (LVLMs)<n>Existing LVLM training paradigms place excessive reliance on the LLM component, giving rise to two critical robustness challenges: language bias and language sensitivity.<n>We propose a novel Self-Critical Inference (SCI) framework that extends Visual Contrastive Decoding by conducting multi-round counterfactual reasoning.
  arXiv  Detail & Related papers  (2026-03-08T14:46:19Z)
• Analyzing Reasoning Consistency in Large Multimodal Models under Cross-Modal Conflicts [74.47786985522762]
  We identify a critical failure mode termed textual inertia, where models tend to blindly adhere to the erroneous text while neglecting conflicting visual evidence.<n>We propose the LogicGraph Perturbation Protocol that structurally injects perturbations into the reasoning chains of diverse LMMs.<n>Results reveal that models successfully self-correct in less than 10% of cases and predominantly succumb to blind textual error propagation.
  arXiv  Detail & Related papers  (2026-01-07T16:39:34Z)
• Robust Uncertainty Quantification for Factual Generation of Large Language Models [22.060021788289202]
  Large language model(LLM) technology has facilitated its integration into various domains of professional and daily life.<n>The persistent challenge of LLM hallucination has emerged as a critical limitation, significantly compromising the reliability and trustworthiness of AI-generated content.<n>This study proposes an uncertainty quantification scenario in the task of generating with multiple facts.
  arXiv  Detail & Related papers  (2026-01-01T14:06:58Z)
• Uncertainty Quantification for Hallucination Detection in Large Language Models: Foundations, Methodology, and Future Directions [28.64896454455385]
  Large language models (LLMs) remain prone to hallucinations that produce plausible but factually incorrect outputs.<n>Uncertainty quantification (UQ) has emerged as a central research direction to address this issue.<n>We examine the role of UQ in hallucination detection, where quantifying uncertainty provides a mechanism for identifying unreliable generations.
  arXiv  Detail & Related papers  (2025-10-14T00:49:04Z)
• Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs [78.09559830840595]
  We present the first systematic study on quantizing diffusion-based language models.<n>We identify the presence of activation outliers, characterized by abnormally large activation values.<n>We implement state-of-the-art PTQ methods and conduct a comprehensive evaluation.
  arXiv  Detail & Related papers  (2025-08-20T17:59:51Z)
• Lost at the Beginning of Reasoning [85.17612793300238]
  We show that the first reasoning step exerts a disproportionately large influence on the final prediction.<n>We propose an efficient sampling strategy that leverages a reward model to identify and retain high-quality first reasoning steps.
  arXiv  Detail & Related papers  (2025-06-27T09:53:57Z)
• A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models [58.32070787537946]
  Chain-of-thought (CoT) reasoning enhances performance of large language models.<n>We present the first comprehensive study of CoT faithfulness in large vision-language models.
  arXiv  Detail & Related papers  (2025-05-29T18:55:05Z)
• The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning [56.574829311863446]
  Chain-of-Thought (CoT) prompting has been widely recognized for its ability to enhance reasoning capabilities in large language models (LLMs)<n>We demonstrate that CoT and its reasoning variants consistently underperform direct answering across varying model scales and benchmark complexities.<n>Our analysis uncovers a fundamental hybrid mechanism of explicit-implicit reasoning driving CoT's performance in pattern-based ICL.
  arXiv  Detail & Related papers  (2025-04-07T13:51:06Z)
• A Unified Virtual Mixture-of-Experts Framework:Enhanced Inference and Hallucination Mitigation in Single-Model System [9.764336669208394]
  Generative models, such as GPT and BERT, have significantly improved performance in tasks like text generation and summarization.<n>However, hallucinations "where models generate non-factual or misleading content" are especially problematic in smaller-scale architectures.<n>We propose a unified Virtual Mixture-of-Experts (MoE) fusion strategy that enhances inference performance and mitigates hallucinations in a single Qwen 1.5 0.5B model.
  arXiv  Detail & Related papers  (2025-04-01T11:38:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.