SIFT: Grounding LLM Reasoning in Contexts via Stickers

• URL: http://arxiv.org/abs/2502.14922v1
• Date: Wed, 19 Feb 2025 17:38:46 GMT
• Title: SIFT: Grounding LLM Reasoning in Contexts via Stickers
• Abstract summary: This paper identifies the misinterpretation of the context can be a significant issue during the reasoning process of large language models.<n>We introduce a novel, post-training approach called **Stick to the Facts (SIFT)** to tackle this.<n>At the core of SIFT lies the *Sticker*, which is generated by the model itself to explicitly emphasize the key information within the context.
• Score: 8.677892927579384
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper identifies the misinterpretation of the context can be a significant issue during the reasoning process of large language models, spanning from smaller models like Llama3.2-3B-Instruct to cutting-edge ones like DeepSeek-R1. For example, in the phrase "10 dollars per kilo," LLMs might not recognize that "per" means "for each," leading to calculation errors. We introduce a novel, post-training approach called **Stick to the Facts (SIFT)** to tackle this. SIFT leverages increasing inference-time compute to ground LLM reasoning in contexts. At the core of SIFT lies the *Sticker*, which is generated by the model itself to explicitly emphasize the key information within the context. Given the curated Sticker, SIFT generates two predictions -- one from the original query and one from the query augmented with the Sticker. If they differ, the Sticker is sequentially refined via *forward* optimization (to better align the extracted facts with the query) and *inverse* generation (to conform with the model's inherent tendencies) for more faithful reasoning outcomes. Studies across diverse models (from 3B to 100B+) and benchmarks (e.g., GSM8K, MATH-500) reveal consistent performance improvements. Notably, SIFT improves the pass@1 accuracy of DeepSeek-R1 on AIME2024 from 78.33% to **85.67**%, establishing a new state-of-the-art in the open-source community. The code is available at https://github.com/zhijie-group/SIFT.

Related papers

• Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Models [51.21958617138889]
  Self-Aware Scheduling learns a lightweight order policy and applies seamlessly to both any-order and semi-autoregressive decoding.<n>On Sudoku with 1B MDM, SAS improves puzzle accuracy from 82.0% (best schedule) to 91.8%, and reaches 97.5% with second-stage fine-tuning.<n>On mathematical reasoning with LLaDA-8B, SAS improves pass@1 on GSM8K from 64% to 76% and on MBPP from 39.5% to 41%.
  arXiv  Detail & Related papers  (2026-06-22T16:32:25Z)
• Introspection Fine-Tuning (IFT): Training Small LLMs to Introspect [0.0]
  We investigate whether small language models can detect and report on perturbations their own internal activations.<n>We first show that the binary detection paradigm used in prior work is confounded in small models, as steering biases the model toward affirmative responses regardless of the question content.<n>We then introduce emphIntrospection Fine-Tuning (IFT), supervised fine-tuning on sentence-localization examples constructed from the model's own perturbed forward passes.
  arXiv  Detail & Related papers  (2026-05-08T05:30:26Z)
• $\ abla$-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space [71.23672814629448]
  $nabla$-Reasoner is an iterative generation framework that integrates differentiable optimization over token logits into the decoding loop.<n>$nabla$-Reasoner achieves over 20% accuracy improvement on a challenging mathematical reasoning benchmark.
  arXiv  Detail & Related papers  (2026-03-05T08:42:54Z)
• Adapting Multilingual Models to Code-Mixed Tasks via Model Merging [9.620539727633274]
  We study model merging as a practical alternative to conventional adaptation strategies for code-mixed NLP.<n>We evaluate our approach for sentence classification (sentiment and hate speech) task in English-Hindi (En-Hi) and English-Spanish (En-Es) using XLM-R and Llama-3.2-1B models.
  arXiv  Detail & Related papers  (2025-10-22T17:16:23Z)
• Supervised Fine-Tuning or Contrastive Learning? Towards Better Multimodal LLM Reranking [56.46309219272326]
  For large language models (LLMs), classification via supervised fine-tuning (SFT) predicts ''yes'' (resp. ''no'') token for relevant (resp. irrelevant) pairs.<n>This divergence raises a central question: which objective is intrinsically better suited to LLM-based reranking, and what mechanism underlies the difference?<n>We conduct a comprehensive comparison and analysis between CL and SFT for reranking, taking the universal multimodal retrieval (UMR) as the experimental playground.
  arXiv  Detail & Related papers  (2025-10-16T16:02:27Z)
• R1-Code-Interpreter: Training LLMs to Reason with Code via Supervised and Reinforcement Learning [14.208804782749793]
  We present R1-Code-Interpreter, an extension of a text-only Large Language Models (LLMs) trained via multi-turn supervised fine-tuning (SFT) and reinforcement learning (RL)<n>R1-Code-Interpreter autonomously generates multiple code queries during step-by-step reasoning.<n>Unlike prior RL work on narrow domains, we find that Code Interpreter training is significantly harder due to high task diversity and expensive code execution.
  arXiv  Detail & Related papers  (2025-05-27T18:47:33Z)
• Shadow-FT: Tuning Instruct Model via Training on Paired Base Model [67.20706292627106]
  Large language models (LLMs) consistently benefit from further fine-tuning on various tasks.<n>We propose a novel Shadow-FT framework to tune the Instruct models by leveraging the corresponding Base models.<n>Our proposed Shadow-FT introduces no additional parameters, is easy to implement, and significantly improves performance.
  arXiv  Detail & Related papers  (2025-05-19T05:16:21Z)
• S$^2$R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning [51.84977135926156]
  We introduce S$2$R, an efficient framework that enhances LLM reasoning by teaching models to self-verify and self-correct during inference.<n>Our results demonstrate that Qwen2.5-math-7B achieves an accuracy improvement from 51.0% to 81.6%, outperforming models trained on an equivalent amount of long-CoT distilled data.
  arXiv  Detail & Related papers  (2025-02-18T13:40:22Z)
• Balancing Truthfulness and Informativeness with Uncertainty-Aware Instruction Fine-Tuning [79.48839334040197]
  Instruction fine-tuning (IFT) can increase the informativeness of large language models (LLMs), but may reduce their truthfulness.<n>In this paper, we empirically demonstrate how unfamiliar knowledge in IFT datasets can negatively affect the truthfulness of LLMs.<n>We introduce two new IFT paradigms, $UNIT_cut$ and $UNIT_ref$, to address this issue.
  arXiv  Detail & Related papers  (2025-02-17T16:10:30Z)
• LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters! [53.84130385074551]
  Large reasoning models (LRMs) tackle complex reasoning problems by following long chain-of-thoughts (Long CoT)<n>We find that a Large Language model (LLM) can effectively learn Long CoT reasoning through data-efficient supervised fine-tuning (SFT) and parameter-efficient low-rank adaptation (LoRA)<n>With just 17k long CoT training samples, the Qwen2.5-32B-Instruct model achieves significant improvements on a wide range of math and coding benchmarks.
  arXiv  Detail & Related papers  (2025-02-11T08:48:48Z)
• Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation [81.18701211912779]
  We introduce an Adaptive Multi-Aspect Retrieval-augmented over KGs (Amar) framework. This method retrieves knowledge including entities, relations, and subgraphs, and converts each piece of retrieved text into prompt embeddings. Our method has achieved state-of-the-art performance on two common datasets.
  arXiv  Detail & Related papers  (2024-12-24T16:38:04Z)
• Disperse-Then-Merge: Pushing the Limits of Instruction Tuning via Alignment Tax Reduction [75.25114727856861]
  Large language models (LLMs) tend to suffer from deterioration at the latter stage ofSupervised fine-tuning process. We introduce a simple disperse-then-merge framework to address the issue. Our framework outperforms various sophisticated methods such as data curation and training regularization on a series of standard knowledge and reasoning benchmarks.
  arXiv  Detail & Related papers  (2024-05-22T08:18:19Z)
• ANLS* -- A Universal Document Processing Metric for Generative Large Language Models [40.94659575657584]
  This paper introduces a new metric for evaluating generative models called ANLS*. The ANLS* metric extends existing ANLS metrics as a drop-in-replacement and is still compatible with previously reported ANLS scores. We also benchmark a novel approach to generate prompts for documents, called SFT, against other prompting techniques such as LATIN.
  arXiv  Detail & Related papers  (2024-02-06T09:50:08Z)
• LaFFi: Leveraging Hybrid Natural Language Feedback for Fine-tuning Language Models [14.087415157225715]
  Fine-tuning Large Language Models (LLMs) adapts a trained model to specific downstream tasks. Supervised Fine-Tuning (SFT) is a common approach, where an LLM is trained to produce desired answers. This paper introduces an alternative to SFT called Natural Language Feedback for Finetuning LLMs (LaFFi)
  arXiv  Detail & Related papers  (2023-12-31T21:18:16Z)
• Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch [72.97553348776425]
  We introduce DARE to set most delta parameters without affecting the abilities of Supervised Fine-Tuning (SFT) LMs. Then, we use DARE as a versatile plug-in to sparsify delta parameters of multiple SFT models and merge them into a single model. We also utilize DARE to create a merged LM that ranks first among models with 7 billion parameters on the Open Leaderboard.
  arXiv  Detail & Related papers  (2023-11-06T13:43:07Z)
• Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study [44.39031420687302]
  Large language models (LLMs) are becoming attractive as few-shot reasoners to solve Natural Language (NL)-related tasks. We try to understand this by designing a benchmark to evaluate the structural understanding capabilities of LLMs. We propose $textitself-augmentation$ for effective structural prompting, such as critical value / range identification.
  arXiv  Detail & Related papers  (2023-05-22T14:23:46Z)
• You can't pick your neighbors, or can you? When and how to rely on retrieval in the $k$NN-LM [65.74934004876914]
  Retrieval-enhanced language models (LMs) condition their predictions on text retrieved from large external datastores. One such approach, the $k$NN-LM, interpolates any existing LM's predictions with the output of a $k$-nearest neighbors model. We empirically measure the effectiveness of our approach on two English language modeling datasets.
  arXiv  Detail & Related papers  (2022-10-28T02:57:40Z)
• Revisiting Self-Training for Few-Shot Learning of Language Model [61.173976954360334]
  Unlabeled data carry rich task-relevant information, they are proven useful for few-shot learning of language model. In this work, we revisit the self-training technique for language model fine-tuning and present a state-of-the-art prompt-based few-shot learner, SFLM.
  arXiv  Detail & Related papers  (2021-10-04T08:51:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.