Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspective

• URL: http://arxiv.org/abs/2605.17967v1
• Date: Mon, 18 May 2026 07:22:54 GMT
• Title: Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspective
• Abstract summary: This paper explores why supervised fine-tuning (SFT) is broadly effective for small-scale deep neural networks, yet can produce inconsistent or even detrimental effects when applied to large language models (LLMs)<n>We find that the evolution of interactions during SFT can effectively explain the inconsistent effectiveness of SFT for LLMs.
• Score: 24.938534291751964
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper explores a scientific question in supervised fine-tuning (SFT): why SFT is broadly effective for small-scale deep neural networks, yet can produce inconsistent or even detrimental effects when applied to large language models (LLMs). Recent advances in interaction-based explanations suggest that interactions between words/tokens provide a faithful metric for quantifying the inference patterns encoded by LLMs. We find that the evolution of interactions during SFT can effectively explain the inconsistent effectiveness of SFT for LLMs. Specifically, we find that (1) SFT primarily removes noise-like interactions, while rarely acquiring reliable new interactions. (2) This denoising stage is extremely brief, after which continued fine-tuning tends to introduce overfitted interactions. We validate these findings across multiple LLMs and datasets. Our findings provide new insights into early stopping and offer practical guidance for LLM training.

Related papers

• Beyond Continuity: Challenges of Context Switching in Multi-Turn Dialogue with LLMs [6.89598705331814]
  We stress-test the multi-turn understanding of Large Language Models (LLMs)<n>We study the following two sub-tasks: detecting whether the user pivots or refines in the current turn, and shortlisting relevant context from previous turns.<n>We evaluate the zero-shot performance of ten LLMs (open-weight, closed-source and reasoning)
  arXiv  Detail & Related papers  (2026-05-10T02:28:23Z)
• Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation [26.91734024759386]
  We argue that the root cause lies in an intent alignment gap rather than intrinsic capability deficits.<n>We propose to decouple intent understanding from task execution through a Mediator-Assistant architecture.
  arXiv  Detail & Related papers  (2026-02-07T03:41:04Z)
• Supervised Fine-Tuning or Contrastive Learning? Towards Better Multimodal LLM Reranking [56.46309219272326]
  For large language models (LLMs), classification via supervised fine-tuning (SFT) predicts ''yes'' (resp. ''no'') token for relevant (resp. irrelevant) pairs.<n>This divergence raises a central question: which objective is intrinsically better suited to LLM-based reranking, and what mechanism underlies the difference?<n>We conduct a comprehensive comparison and analysis between CL and SFT for reranking, taking the universal multimodal retrieval (UMR) as the experimental playground.
  arXiv  Detail & Related papers  (2025-10-16T16:02:27Z)
• When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models [16.12256921806929]
  Large language models (LLMs) have achieved impressive performance across natural language processing (NLP) tasks.<n>As real-world applications increasingly demand longer context windows, continued pretraining and supervised fine-tuning (SFT) on long-context data has become a common approach.<n>We systematically investigate how SFT data length influences LLM behavior on short-context tasks.<n>We find that long-context SFT improves short-context performance, contrary to the commonly observed degradation from long-context pretraining.
  arXiv  Detail & Related papers  (2025-09-23T07:55:38Z)
• The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs [66.17068546293487]
  Large vision-language models (VLMs) increasingly adopt post-training techniques such as long chain-of-thought (CoT) supervised fine-tuning (SFT) and reinforcement learning (RL) to elicit sophisticated reasoning.<n>We present a systematic investigation into the distinct roles and interplay of long-CoT SFT and RL across multiple multimodal reasoning benchmarks.<n>We find that SFT improves performance on difficult questions by in-depth, structured reasoning, but introduces verbosity and degrades performance on simpler ones.
  arXiv  Detail & Related papers  (2025-07-10T09:05:49Z)
• Why Reinforcement Fine-Tuning Enables MLLMs Preserve Prior Knowledge Better: A Data Perspective [98.45690529036848]
  Post-training algorithms such as Supervised Fine-Tuning (SFT) and Reinforcement Fine-Tuning (RFT) are widely used to adapt multimodal large language models to downstream tasks.<n>While effective at task adaptation, their impact on prior knowledge remains unclear.
  arXiv  Detail & Related papers  (2025-06-30T04:15:01Z)
• Balancing Truthfulness and Informativeness with Uncertainty-Aware Instruction Fine-Tuning [79.48839334040197]
  Instruction fine-tuning (IFT) can increase the informativeness of large language models (LLMs), but may reduce their truthfulness.<n>In this paper, we empirically demonstrate how unfamiliar knowledge in IFT datasets can negatively affect the truthfulness of LLMs.<n>We introduce two new IFT paradigms, $UNIT_cut$ and $UNIT_ref$, to address this issue.
  arXiv  Detail & Related papers  (2025-02-17T16:10:30Z)
• From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuning [91.79567270986901]
  Large Language Models (LLMs) tend to prioritize adherence to user prompts over providing veracious responses.<n>Recent works propose to employ supervised fine-tuning (SFT) to mitigate the sycophancy issue.<n>We propose a novel supervised pinpoint tuning (SPT), where the region-of-interest modules are tuned for a given objective.
  arXiv  Detail & Related papers  (2024-09-03T07:01:37Z)
• LaFFi: Leveraging Hybrid Natural Language Feedback for Fine-tuning Language Models [14.087415157225715]
  Fine-tuning Large Language Models (LLMs) adapts a trained model to specific downstream tasks. Supervised Fine-Tuning (SFT) is a common approach, where an LLM is trained to produce desired answers. This paper introduces an alternative to SFT called Natural Language Feedback for Finetuning LLMs (LaFFi)
  arXiv  Detail & Related papers  (2023-12-31T21:18:16Z)
• Instruction Tuning for Large Language Models: A Survey [52.86322823501338]
  We make a systematic review of the literature, including the general methodology of supervised fine-tuning (SFT)<n>We also review the potential pitfalls of SFT along with criticism against it, along with efforts pointing out current deficiencies of existing strategies.
  arXiv  Detail & Related papers  (2023-08-21T15:35:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.