MAC-Tuning: LLM Multi-Compositional Problem Reasoning with Enhanced Knowledge Boundary Awareness

• URL: http://arxiv.org/abs/2504.21773v1
• Date: Wed, 30 Apr 2025 16:17:53 GMT
• Title: MAC-Tuning: LLM Multi-Compositional Problem Reasoning with Enhanced Knowledge Boundary Awareness
• Abstract summary: We introduce a novel method, Multiple Answers and Confidence Stepwise Tuning (MAC-Tuning), that separates the learning of answer prediction and confidence estimation during fine-tuning on instruction data.<n>Our method outperforms baselines by up to 25% in average precision.
• Score: 1.5601146551243388
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the widespread application of large language models (LLMs), the issue of generating non-existing facts, known as hallucination, has garnered increasing attention. Previous research in enhancing LLM confidence estimation mainly focuses on the single problem setting. However, LLM awareness of its internal parameterized knowledge boundary under the more challenging multi-problem setting, which requires answering multiple problems accurately simultaneously, remains underexplored. To bridge this gap, we introduce a novel method, Multiple Answers and Confidence Stepwise Tuning (MAC-Tuning), that separates the learning of answer prediction and confidence estimation during fine-tuning on instruction data. Extensive experiments demonstrate that our method outperforms baselines by up to 25% in average precision.

Related papers

• Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitation [49.86232017439639]
  We introduce Multi-Output Augmented Behavioral Cloning (MA-BC)<n>MA-BC partitions divergent expert data while pooling state-action pairs where no behavior conflict is observed.<n>We establish a novel lower bound for multi-objective imitation learning, demonstrating that MA-BC is minimax optimal.
  arXiv  Detail & Related papers  (2026-05-12T11:49:08Z)
• Investigating LLM's Problem Solving Capability -- a Study on Statics Questions [0.0]
  Large Language Models (LLMs) have rapidly influenced many aspects of society, particularly education.<n>This study adopts a model distillation process to evaluate LLM capabilities in solving statics problems.<n> Experimental results show that while LLMs perform well on text-only statics problems, their accuracy decreases when diagrams are introduced.
  arXiv  Detail & Related papers  (2026-04-30T20:17:09Z)
• Evaluating and Calibrating LLM Confidence on Questions with Multiple Correct Answers [11.832359446987551]
  We show that existing training-free methods break down in the presence of multiple valid answers.<n>We propose Semantic Confidence Aggregation (SCA), which aggregates confidence over multiple high-probability sampled responses.<n> SCA achieves state-of-the-art calibration performance under mixed-answer settings while preserving strong calibration on single-answer questions.
  arXiv  Detail & Related papers  (2026-02-08T07:03:27Z)
• Fine-Tuning vs. RAG for Multi-Hop Question Answering with Novel Knowledge [7.716590111773082]
  We compare parametric and non-parametric knowledge injection methods for open-domain multi-hop question answering.<n>We evaluate unsupervised fine-tuning, supervised fine-tuning, and retrieval-augmented generation.<n>Retrieval-augmented generation yields substantial and consistent improvements when answering questions that rely on temporally novel information.
  arXiv  Detail & Related papers  (2026-01-11T20:24:25Z)
• Robust Uncertainty Quantification for Factual Generation of Large Language Models [22.060021788289202]
  Large language model(LLM) technology has facilitated its integration into various domains of professional and daily life.<n>The persistent challenge of LLM hallucination has emerged as a critical limitation, significantly compromising the reliability and trustworthiness of AI-generated content.<n>This study proposes an uncertainty quantification scenario in the task of generating with multiple facts.
  arXiv  Detail & Related papers  (2026-01-01T14:06:58Z)
• Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Tasks [54.31998314008198]
  Large Language Models (LLMs) excel in reasoning tasks requiring a single correct answer, but they perform poorly in multi-solution tasks.<n>We attribute this limitation to textbfreasoning overconfidence: a tendency to express undue certainty in an incomplete solution set.<n>We propose the textbfcognitive-rigidity hypothesis, which posits that overconfidence arises when the reasoning process prematurely converges on a narrow set of thought paths.
  arXiv  Detail & Related papers  (2025-12-01T14:35:06Z)
• Can Multiple Responses from an LLM Reveal the Sources of Its Uncertainty? [11.309445539128733]
  Large language models (LLMs) have delivered significant breakthroughs across diverse domains but can still produce unreliable or misleading outputs.<n>We show that, when an LLM is uncertain, the patterns of disagreement among its multiple generated responses contain rich clues about the underlying cause of uncertainty.
  arXiv  Detail & Related papers  (2025-08-28T20:14:35Z)
• ReliableMath: Benchmark of Reliable Mathematical Reasoning on Large Language Models [70.33764118171463]
  Large Language Models (LLMs) tend to fabricate unreliable responses when confronted with problems that are unsolvable or beyond their capability.<n>We develop a ReliableMath dataset which incorporates open-source solvable problems and high-quality unsolvable problems.<n>LLMs fail to directly identify unsolvable problems and always generate fabricated responses.
  arXiv  Detail & Related papers  (2025-07-03T19:19:44Z)
• MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibration [2.1824579248418017]
  We present MMBoundary, a novel framework that advances the knowledge boundary awareness of MLLMs through reasoning step confidence calibration.<n>In addition to supervised fine-tuning, we introduce a reinforcement learning stage with multiple reward functions for further aligning model knowledge.<n> Empirical results show that MMBoundary significantly outperforms existing methods across diverse domain datasets and metrics.
  arXiv  Detail & Related papers  (2025-05-29T08:14:40Z)
• Reinforcing Question Answering Agents with Minimalist Policy Gradient Optimization [80.09112808413133]
  Mujica is a planner that decomposes questions into acyclic graph of subquestions and a worker that resolves questions via retrieval and reasoning.<n>MyGO is a novel reinforcement learning method that replaces traditional policy updates with gradient Likelihood Maximum Estimation.<n> Empirical results across multiple datasets demonstrate the effectiveness of MujicaMyGO in enhancing multi-hop QA performance.
  arXiv  Detail & Related papers  (2025-05-20T18:33:03Z)
• Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding [48.92310906093414]
  We introduce a novel approach for calibrating uncertainty quantification (UQ) tailored for multi-modal large language models (LLMs)<n>We leverage cross-modal consistency in addition to self-consistency to improve the calibration of the multi-modal models.<n>We evaluate the proposed approach across multiple multi-modal tasks, such as medical question answering (Slake) and visual question answering (VQAv2), considering multi-modal models such as LLaVA-Med and LLaVA.
  arXiv  Detail & Related papers  (2025-04-30T19:19:21Z)
• Accommodate Knowledge Conflicts in Retrieval-augmented LLMs: Towards Reliable Response Generation in the Wild [11.058848731627233]
  Large language models (LLMs) have advanced information retrieval systems.<n>LLMs often face knowledge conflicts between internal memory and retrievaled external information.<n>We propose Swin-VIB, a novel framework that integrates a pipeline of variational information bottleneck models into adaptive augmentation of retrieved information.
  arXiv  Detail & Related papers  (2025-04-17T14:40:31Z)
• Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perception [58.62352010928591]
  Large language models (LLMs) exhibit impressive performance across diverse tasks but often struggle to accurately gauge their knowledge boundaries.<n>This paper explores leveraging LLMs' internal states to enhance their perception of knowledge boundaries from efficiency and risk perspectives.
  arXiv  Detail & Related papers  (2025-02-17T11:11:09Z)
• Enhancing Multi-Agent Consensus through Third-Party LLM Integration: Analyzing Uncertainty and Mitigating Hallucinations in Large Language Models [1.4582633500696451]
  Large Language Models (LLMs) still face challenges when dealing with complex reasoning tasks. This paper proposes a new method that integrates different LLMs to expand the knowledge boundary. Experiments on arithmetic datasets have validated the effectiveness of the method.
  arXiv  Detail & Related papers  (2024-11-25T08:42:33Z)
• Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning [104.27224674122313]
  Fine-tuning MLLM has become a common practice to improve performance on specific downstream tasks. To balance the trade-off between generalization and specialization, we propose measuring the parameter importance for both pre-trained and fine-tuning distributions.
  arXiv  Detail & Related papers  (2024-11-17T01:16:37Z)
• Exploring Knowledge Boundaries in Large Language Models for Retrieval Judgment [56.87031484108484]
  Large Language Models (LLMs) are increasingly recognized for their practical applications. Retrieval-Augmented Generation (RAG) tackles this challenge and has shown a significant impact on LLMs. By minimizing retrieval requests that yield neutral or harmful results, we can effectively reduce both time and computational costs.
  arXiv  Detail & Related papers  (2024-11-09T15:12:28Z)
• Do LLMs estimate uncertainty well in instruction-following? [9.081508933326644]
  Large language models (LLMs) could be valuable personal AI agents across various domains, provided they can precisely follow user instructions. We present the first systematic evaluation of the uncertainty estimation abilities of LLMs in the context of instruction-following. Our findings show that existing uncertainty methods struggle, particularly when models make subtle errors in instruction following.
  arXiv  Detail & Related papers  (2024-10-18T16:32:10Z)
• Understanding the Relationship between Prompts and Response Uncertainty in Large Language Models [55.332004960574004]
  Large language models (LLMs) are widely used in decision-making, but their reliability, especially in critical tasks like healthcare, is not well-established.<n>This paper investigates how the uncertainty of responses generated by LLMs relates to the information provided in the input prompt.<n>We propose a prompt-response concept model that explains how LLMs generate responses and helps understand the relationship between prompts and response uncertainty.
  arXiv  Detail & Related papers  (2024-07-20T11:19:58Z)
• UBENCH: Benchmarking Uncertainty in Large Language Models with Multiple Choice Questions [10.28688988951815]
  UBENCH is a benchmark for evaluating large language models. It includes 3,978 multiple-choice questions covering knowledge, language, understanding, and reasoning abilities. We also evaluate the reliability of 15 popular LLMs, finding GLM4 to be the most outstanding.
  arXiv  Detail & Related papers  (2024-06-18T16:50:38Z)
• MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning [36.400896909161006]
  We develop systems that proactively ask questions to gather more information and respond reliably. We introduce a benchmark - MediQ - to evaluate question-asking ability in LLMs.
  arXiv  Detail & Related papers  (2024-06-03T01:32:52Z)
• Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedback [14.120154004011084]
  Large Language Models (LLMs) often generate erroneous outputs, known as hallucinations. We present a novel alignment framework called Reinforcement Learning from Knowledge Feedback (RLKF)
  arXiv  Detail & Related papers  (2024-03-27T08:39:56Z)
• ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent [50.508669199496474]
  We develop a ReAct-style LLM agent with the ability to reason and act upon external knowledge. We refine the agent through a ReST-like method that iteratively trains on previous trajectories. Starting from a prompted large model and after just two iterations of the algorithm, we can produce a fine-tuned small model.
  arXiv  Detail & Related papers  (2023-12-15T18:20:15Z)
• Improving Open Information Extraction with Large Language Models: A Study on Demonstration Uncertainty [52.72790059506241]
  Open Information Extraction (OIE) task aims at extracting structured facts from unstructured text. Despite the potential of large language models (LLMs) like ChatGPT as a general task solver, they lag behind state-of-the-art (supervised) methods in OIE tasks.
  arXiv  Detail & Related papers  (2023-09-07T01:35:24Z)
• Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation [109.8527403904657]
  We show that large language models (LLMs) possess unwavering confidence in their knowledge and cannot handle the conflict between internal and external knowledge well. Retrieval augmentation proves to be an effective approach in enhancing LLMs' awareness of knowledge boundaries. We propose a simple method to dynamically utilize supporting documents with our judgement strategy.
  arXiv  Detail & Related papers  (2023-07-20T16:46:10Z)
• Uncertainty Estimation by Fisher Information-based Evidential Deep Learning [61.94125052118442]
  Uncertainty estimation is a key factor that makes deep learning reliable in practical applications. We propose a novel method, Fisher Information-based Evidential Deep Learning ($mathcalI$-EDL) In particular, we introduce Fisher Information Matrix (FIM) to measure the informativeness of evidence carried by each sample, according to which we can dynamically reweight the objective loss terms to make the network more focused on the representation learning of uncertain classes.
  arXiv  Detail & Related papers  (2023-03-03T16:12:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.