ToM-LM: Delegating Theory of Mind Reasoning to External Symbolic Executors in Large Language Models

• URL: http://arxiv.org/abs/2404.15515v3
• Date: Wed, 26 Jun 2024 15:57:22 GMT
• Title: ToM-LM: Delegating Theory of Mind Reasoning to External Symbolic Executors in Large Language Models
• Authors: Weizhi Tang, Vaishak Belle,
• Abstract summary: Theory of Mind (ToM) refers to the ability of individuals to attribute mental states to others. Large Language Models (LLMs) have shown some promise with ToM ability, but they still struggle with complex ToM reasoning.
• Score: 5.455744338342196
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Theory of Mind (ToM) refers to the ability of individuals to attribute mental states to others. While Large Language Models (LLMs) have shown some promise with ToM ability, they still struggle with complex ToM reasoning. Our approach leverages an external symbolic executor, specifically the SMCDEL model checker, and fine-tuning to improve the ToM reasoning ability of LLMs. In our approach, an LLM is first fine-tuned through pairs of natural language and symbolic formulation representation of ToM problems and is then instructed to generate the symbolic formulation with a one-shot in-context example. The generated symbolic formulation is then executed by the SMCDEL model checker to perform transparent and verifiable ToM reasoning and give the final result. We demonstrate that our approach, ToM-LM, shows a significant improvement over all the constructed baselines. Our study proposes a novel view about externalizing a particular component of ToM reasoning, mainly reasoning about beliefs, and suggests generalizing it to other aspects of ToM reasoning.

Related papers

• Constrained Reasoning Chains for Enhancing Theory-of-Mind in Large Language Models [39.81210971002642]
  Theory-of-Mind (ToM) ability possessed by Large Language Models (LLMs) has been shown to be limited. We propose Constrained Chain-of-ToM (CCoToM) that leverages domain knowledge and causal relations between ToM dimensions to address limitations. CCoToM consistently outperforms previous state-of-the-art methods by large margins across all datasets used.
  arXiv  Detail & Related papers  (2024-09-20T13:27:11Z)
• Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models [51.91448005607405]
  We evaluate key human ToM precursors by annotating characters' perceptions on ToMi and FANToM. We present PercepToM, a novel ToM method leveraging LLMs' strong perception inference capability while supplementing their limited perception-to-belief inference.
  arXiv  Detail & Related papers  (2024-07-08T14:58:29Z)
• Through the Theory of Mind's Eye: Reading Minds with Multimodal Video Large Language Models [52.894048516550065]
  We develop a pipeline for multimodal ToM reasoning using video and text. We also enable explicit ToM reasoning by retrieving key frames for answering a ToM question.
  arXiv  Detail & Related papers  (2024-06-19T18:24:31Z)
• Do LLMs Exhibit Human-Like Reasoning? Evaluating Theory of Mind in LLMs for Open-Ended Responses [11.121931601655174]
  Theory of Mind (ToM) reasoning entails recognizing that other individuals possess their own intentions, emotions, and thoughts. Large language models (LLMs) excel in tasks such as summarization, question answering, and translation. Despite advancements, the extent to which LLMs truly understand ToM reasoning remains inadequately explored in open-ended scenarios.
  arXiv  Detail & Related papers  (2024-06-09T05:57:59Z)
• Zero, Finite, and Infinite Belief History of Theory of Mind Reasoning in Large Language Models [5.455744338342196]
  Large Language Models (LLMs) have recently shown a promise and emergence of Theory of Mind (ToM) ability. We propose a novel concept, taxonomy, and framework, the ToM reasoning with Zero, Finite, and Infinite Belief History. We have evaluated six LLMs with this game and found their performance on Zero Belief History is consistently better than on Finite Belief History.
  arXiv  Detail & Related papers  (2024-06-07T10:04:39Z)
• Fact :Teaching MLLMs with Faithful, Concise and Transferable Rationales [102.54274021830207]
  We introduce Fact, a novel paradigm designed to generate multimodal rationales that are faithful, concise, and transferable for teaching MLLMs. We filter rationales that can be transferred to end-to-end paradigms from programming paradigms to guarantee transferability. Our approach also reduces hallucinations owing to its high correlation between images and text.
  arXiv  Detail & Related papers  (2024-04-17T07:20:56Z)
• Views Are My Own, but Also Yours: Benchmarking Theory of Mind Using Common Ground [6.868969074841911]
  We introduce the first ToM dataset based on naturally occurring spoken dialogs, Common-ToM, and show that LMs struggle to demonstrate ToM. We then show that integrating a simple, explicit representation of beliefs improves LM performance on Common-ToM.
  arXiv  Detail & Related papers  (2024-03-04T20:07:17Z)
• MMToM-QA: Multimodal Theory of Mind Question Answering [80.87550820953236]
  Theory of Mind (ToM) is an essential ingredient for developing machines with human-level social intelligence. Recent machine learning models, particularly large language models, seem to show some aspects of ToM understanding. Human ToM, on the other hand, is more than video or text understanding. People can flexibly reason about another person's mind based on conceptual representations extracted from any available data.
  arXiv  Detail & Related papers  (2024-01-16T18:59:24Z)
• Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilities [63.90227161974381]
  SimToM is a novel prompting framework inspired by Simulation Theory's notion of perspective-taking. Our approach, which requires no additional training and minimal prompt-tuning, shows substantial improvement over existing methods.
  arXiv  Detail & Related papers  (2023-11-16T22:49:27Z)
• Theory of Mind in Large Language Models: Examining Performance of 11 State-of-the-Art models vs. Children Aged 7-10 on Advanced Tests [1.099532646524593]
  We test 11 base- and instruction-tuned Large Language Models (LLMs) on capabilities relevant to Theory of Mind (ToM) We find that instruction-tuned LLMs from the GPT family outperform other models, and often also children. We suggest that the interlinked evolution and development of language and ToM may help explain what instruction-tuning adds.
  arXiv  Detail & Related papers  (2023-10-31T09:55:07Z)
• FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions [94.61530480991627]
  Theory of mind evaluations currently focus on testing models using passive narratives that inherently lack interactivity. We introduce FANToM, a new benchmark designed to stress-test ToM within information-asymmetric conversational contexts via question answering.
  arXiv  Detail & Related papers  (2023-10-24T00:24:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.