LP-LM: No Hallucinations in Question Answering with Logic Programming

• URL: http://arxiv.org/abs/2502.09212v1
• Date: Thu, 13 Feb 2025 11:48:31 GMT
• Title: LP-LM: No Hallucinations in Question Answering with Logic Programming
• Abstract summary: Large language models (LLMs) are able to generate human-like responses to user queries.<n>This paper introduces LP-LM, a system that grounds answers to questions in known facts contained in a knowledge base (KB)<n>By leveraging DCG and tabling, LP-LM runs in linear time in the size of input sentences for sufficiently many grammar rules.
• Score: 0.30693357740321775
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are able to generate human-like responses to user queries. However, LLMs exhibit inherent limitations, especially because they hallucinate. This paper introduces LP-LM, a system that grounds answers to questions in known facts contained in a knowledge base (KB), facilitated through semantic parsing in Prolog, and always produces answers that are reliable. LP-LM generates a most probable constituency parse tree along with a corresponding Prolog term for an input question via Prolog definite clause grammar (DCG) parsing. The term is then executed against a KB of natural language sentences also represented as Prolog terms for question answering. By leveraging DCG and tabling, LP-LM runs in linear time in the size of input sentences for sufficiently many grammar rules. Performing experiments comparing LP-LM with current well-known LLMs in accuracy, we show that LLMs hallucinate on even simple questions, unlike LP-LM.

Related papers

• Think Fast, Talk Smart: Partitioning Deterministic and Neural Computation for Structured Health Text Generation [0.0764671395172401]
  Large language models (LLMs) are increasingly being used to generate health text from structured records such as wearable time series, biomarkers, vitals, and care-management logs.<n>We ask which responsibilities in structured health generation should be deterministic runtime rather than LLM prompting.<n>We introduce Think Fast, Talk Smart, a sleep-health insight pipeline in which deterministic code performs recurring analysis before one bounded LLM writer call.
  arXiv  Detail & Related papers  (2026-05-28T09:16:21Z)
• Logical Consistency of Large Language Models in Fact-checking [6.286017217366497]
  Large language models (LLMs) have demonstrated significant success in performing varied natural language tasks.<n>Despite their impressive ability to generate human-like texts, LLMs are infamous for their inconsistent responses.
  arXiv  Detail & Related papers  (2024-12-20T17:42:25Z)
• A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagation [72.93327642336078]
  We propose Belief Tree Propagation (BTProp), a probabilistic framework for hallucination detection.<n>BTProp introduces a belief tree of logically related statements by decomposing a parent statement into child statements.<n>Our method improves baselines by 3%-9% (evaluated by AUROC and AUC-PR) on multiple hallucination detection benchmarks.
  arXiv  Detail & Related papers  (2024-06-11T05:21:37Z)
• Log Parsing using LLMs with Self-Generated In-Context Learning and Self-Correction [15.93927602769091]
  Recent emergence of large language models (LLMs) has demonstrated strong abilities in understanding natural language and code.<n>Ada is an effective and adaptive log parsing framework using LLMs with self-generated in-context learning (SG-ICL) and self-correction.<n>Ada outperforms state-of-the-art methods across all metrics, even in zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-06-05T15:31:43Z)
• Arithmetic Reasoning with LLM: Prolog Generation & Permutation [2.1867261071129125]
  We show that Prolog-based arithmetic problem-solving outperforms CoT generation in the GSM8K benchmark. We propose to permute the ground truth predicates for more robust LLM training via data augmentation.
  arXiv  Detail & Related papers  (2024-05-28T07:13:25Z)
• LLMParser: An Exploratory Study on Using Large Language Models for Log Parsing [8.647406441990396]
  We study the potential of using Large Language Models (LLMs) for log parsing and propose an LLM-based log based on generative inferences and few-shot tuning. We find that smaller LLMs may be more effective than more complex LLMs; for instance where Flan-T5-base achieves comparable results as LLaMA-7B with a shorter time. We also find that using LLMs pre-trained using logs from other systems does not always improve parsing accuracy.
  arXiv  Detail & Related papers  (2024-04-27T20:34:29Z)
• How Proficient Are Large Language Models in Formal Languages? An In-Depth Insight for Knowledge Base Question Answering [52.86931192259096]
  Knowledge Base Question Answering (KBQA) aims to answer natural language questions based on facts in knowledge bases. Recent works leverage the capabilities of large language models (LLMs) for logical form generation to improve performance.
  arXiv  Detail & Related papers  (2024-01-11T09:27:50Z)
• Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs [95.07757789781213]
  Two lines of approaches are adopted for complex reasoning with LLMs. One line of work prompts LLMs with various reasoning structures, while the structural outputs can be naturally regarded as intermediate reasoning steps. The other line of work adopt LLM-free declarative solvers to do the reasoning task, rendering higher reasoning accuracy but lacking interpretability due to the black-box nature of the solvers. We present a simple extension to the latter line of work. Specifically, we showcase that the intermediate search logs generated by Prolog interpreters can be accessed and interpreted into human-readable reasoning.
  arXiv  Detail & Related papers  (2023-11-16T11:26:21Z)
• LILAC: Log Parsing using LLMs with Adaptive Parsing Cache [38.04960745458878]
  We propose LILAC, the first practical log parsing framework using large language models (LLMs) with adaptive parsing cache. LLMs's lack of specialized log parsing capabilities currently hinders their accuracy in parsing. We show LILAC outperforms state-of-the-art methods by 69.5% in terms of the average F1 score of template accuracy.
  arXiv  Detail & Related papers  (2023-10-03T04:46:59Z)
• Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning [101.26814728062065]
  Large Language Models (LLMs) have shown human-like reasoning abilities but still struggle with complex logical problems. This paper introduces a novel framework, Logic-LM, which integrates LLMs with symbolic solvers to improve logical problem-solving.
  arXiv  Detail & Related papers  (2023-05-20T22:25:38Z)
• Statistical Knowledge Assessment for Large Language Models [79.07989821512128]
  Given varying prompts regarding a factoid question, can a large language model (LLM) reliably generate factually correct answers? We propose KaRR, a statistical approach to assess factual knowledge for LLMs. Our results reveal that the knowledge in LLMs with the same backbone architecture adheres to the scaling law, while tuning on instruction-following data sometimes compromises the model's capability to generate factually correct text reliably.
  arXiv  Detail & Related papers  (2023-05-17T18:54:37Z)
• Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback [127.75419038610455]
  Large language models (LLMs) are able to generate human-like, fluent responses for many downstream tasks. This paper proposes a LLM-Augmenter system, which augments a black-box LLM with a set of plug-and-play modules.
  arXiv  Detail & Related papers  (2023-02-24T18:48:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.