Can Large Language Models generalize analogy solving like people can?

• URL: http://arxiv.org/abs/2411.02348v2
• Date: Tue, 11 Mar 2025 19:51:32 GMT
• Title: Can Large Language Models generalize analogy solving like people can?
• Authors: Claire E. Stevenson, Alexandra Pafford, Han L. J. van der Maas, Melanie Mitchell,
• Abstract summary: In people, the ability to solve analogies such as "body : feet :: table :?" emerges in childhood.<n>Recent research shows that large language models (LLMs) can solve various forms of analogies.
• Score: 46.02074643846298
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: When we solve an analogy we transfer information from a known context to a new one through abstract rules and relational similarity. In people, the ability to solve analogies such as "body : feet :: table : ?" emerges in childhood, and appears to transfer easily to other domains, such as the visual domain "( : ) :: < : ?". Recent research shows that large language models (LLMs) can solve various forms of analogies. However, can LLMs generalize analogy solving to new domains like people can? To investigate this, we had children, adults, and LLMs solve a series of letter-string analogies (e.g., a b : a c :: j k : ?) in the Latin alphabet, in a near transfer domain (Greek alphabet), and a far transfer domain (list of symbols). As expected, children and adults easily generalized their knowledge to unfamiliar domains, whereas LLMs did not. This key difference between human and AI performance is evidence that these LLMs still struggle with robust human-like analogical transfer.

Related papers

• Types of Relations: Defining Analogies with Category Theory [0.0]
  In this paper, we study features of a domain that are important for constructing analogies.<n>We do so by formalizing knowledge domains as categories.<n>We also show how functors, pullbacks, and pushouts can be used to define an analogy.
  arXiv  Detail & Related papers  (2025-05-26T10:22:44Z)
• Is analogy enough to draw novel adjective-noun inferences? [9.30694340695458]
  We study whether inferences can instead be derived by analogy to known inferences, without need for composition. We find that there are novel combinations for which both humans and LLMs derive convergent inferences but which are not well handled by analogy.
  arXiv  Detail & Related papers  (2025-03-31T16:41:16Z)
• Do Large Language Models Solve ARC Visual Analogies Like People Do? [0.0]
  We compared human and large language model (LLM) performance on a new child-friendly set of ARC items. Results show that both children and adults outperform most LLMs on these tasks.
  arXiv  Detail & Related papers  (2024-03-13T09:48:13Z)
• ParallelPARC: A Scalable Pipeline for Generating Natural-Language Analogies [16.92480305308536]
  We develop a pipeline for creating complex, paragraph-based analogies. We publish a gold-set, validated by humans, and a silver-set, generated automatically. We demonstrate that our silver-set is useful for training models.
  arXiv  Detail & Related papers  (2024-03-02T08:53:40Z)
• Divergences between Language Models and Human Brains [63.405788999891335]
  Recent research has hinted that brain signals can be effectively predicted using internal representations of language models (LMs) We show that there are clear differences in how LMs and humans represent and use language. We identify two domains that are not captured well by LMs: social/emotional intelligence and physical commonsense.
  arXiv  Detail & Related papers  (2023-11-15T19:02:40Z)
• Do large language models solve verbal analogies like children do? [10.616401727158626]
  This paper investigates whether large language models (LLMs) solve verbal analogies in A:B::C:? form using associations, similar to what children do. We use verbal analogies extracted from an online adaptive learning environment, where 14,002 7-12 year-olds from the Netherlands solved 622 analogies in Dutch. We conclude that the LLMs we tested indeed tend to solve verbal analogies by association with C like children do.
  arXiv  Detail & Related papers  (2023-10-31T11:49:11Z)
• StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding [72.38872974837462]
  We evaluate the ability to identify and generate analogies by constructing a first-of-its-kind large-scale story-level analogy corpus. textscStory Analogy contains 24K story pairs from diverse domains with human annotations on two similarities from the extended Structure-Mapping Theory. We observe that the data in textscStory Analogy can improve the quality of analogy generation in large language models.
  arXiv  Detail & Related papers  (2023-10-19T16:29:23Z)
• On the Relationship between Sentence Analogy Identification and Sentence Structure Encoding in Large Language Models [7.716762867270514]
  We look at how Large Language Models' abilities to capture sentence analogies vary with their abilities to encode syntactic and semantic structures. We find that the LLMs which capture syntactic structures better, also have higher abilities in identifying sentence analogies.
  arXiv  Detail & Related papers  (2023-10-11T18:59:48Z)
• Why Do We Need Neuro-symbolic AI to Model Pragmatic Analogies? [6.8107181513711055]
  A hallmark of intelligence is the ability to use a familiar domain to make inferences about a less familiar domain, known as analogical reasoning. We discuss analogies at four distinct levels of complexity: lexical analogies, syntactic analogies, semantic analogies, and pragmatic analogies. We employ Neuro-symbolic AI techniques that combine statistical and symbolic AI, informing the representation of unstructured text to highlight and augment relevant content, provide abstraction and guide the mapping process.
  arXiv  Detail & Related papers  (2023-08-02T21:13:38Z)
• Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations [62.61495090463084]
  Large language models (LLMs) are trained to imitate humans to explain human decisions. We evaluate whether an explanation can enable humans to precisely infer the model's outputs on diverse counterfactuals. We found that LLM's explanations have low precision and that precision does not correlate with plausibility.
  arXiv  Detail & Related papers  (2023-07-17T17:41:47Z)
• Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning [101.26814728062065]
  Large Language Models (LLMs) have shown human-like reasoning abilities but still struggle with complex logical problems. This paper introduces a novel framework, Logic-LM, which integrates LLMs with symbolic solvers to improve logical problem-solving.
  arXiv  Detail & Related papers  (2023-05-20T22:25:38Z)
• ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base [51.777618249271725]
  ANALOGYKB is a million-scale analogy knowledge base derived from existing knowledge graphs (KGs) It identifies two types of analogies from the KGs: 1) analogies of the same relations, which can be directly extracted from the KGs, and 2) analogies of analogous relations, which are identified with a selection and filtering pipeline enabled by large language models (LLMs)
  arXiv  Detail & Related papers  (2023-05-10T09:03:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.