Benchmarking graph construction by large language models for coherence-driven inference

• URL: http://arxiv.org/abs/2502.13953v2
• Date: Wed, 20 Aug 2025 13:10:14 GMT
• Title: Benchmarking graph construction by large language models for coherence-driven inference
• Authors: Steve Huntsman, Jewell Thomas,
• Abstract summary: We benchmark the ability of large language models to reconstruct coherence graphs.<n>Coherence-driven inference on consistency evaluations by LLMs may advance machine cognition capabilities.
• Score: 3.522062800701924
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We devise an algorithm to generate propositions that objectively instantiate graphs supporting coherence-driven inference. We also benchmark the ability of large language models (LLMs) to reconstruct coherence graphs from (a simple transformation of) propositions expressed in natural language, with promising results from a single prompt to reasoning-optimized LLMs. For example, o1/3/4-mini achieve perfect reconstruction half of the time on sparse graphs. Coherence-driven inference on consistency evaluations by LLMs may advance machine cognition capabilities.

Related papers

• Combating Spurious Correlations in Graph Interpretability via Self-Reflection [4.81017678027464]
  Interpretable graph learning is a popular research topic in machine learning.<n>One of the most challenging is the Spurious-Motif benchmark, introduced at ICLR 2022.<n>We propose a self-reflection framework that can be integrated with existing interpretable graph learning methods.
  arXiv  Detail & Related papers  (2026-01-16T06:31:16Z)
• Improving LLM Reasoning with Homophily-aware Structural and Semantic Text-Attributed Graph Compression [55.51959317490934]
  Large language models (LLMs) have demonstrated promising capabilities in Text-Attributed Graph (TAG) understanding.<n>We argue that graphs inherently contain rich structural and semantic information, and that their effective exploitation can unlock potential gains in LLMs reasoning performance.<n>We propose Homophily-aware Structural and Semantic Compression for LLMs (HS2C), a framework centered on exploiting graph homophily.
  arXiv  Detail & Related papers  (2026-01-13T03:35:18Z)
• TinyGraphEstimator: Adapting Lightweight Language Models for Graph Structure Inference [0.48302896549293584]
  This paper investigates whether compact transformer-based language models can infer graph-theoretic parameters directly from graph representations.<n>We introduce the TinyGraphEstimator dataset - a collection of connected graphs generated from multiple random graph models.<n>We evaluate several small open models on their ability to predict key graph parameters such as density, clustering, and chromatic number.
  arXiv  Detail & Related papers  (2025-10-09T20:47:07Z)
• CoT Referring: Improving Referring Expression Tasks with Grounded Reasoning [67.18702329644526]
  CoT Referring enhances model reasoning across modalities through a structured, chain-of-thought training data structure.<n>We restructure the training data to enforce a new output form, providing new annotations for existing datasets.<n>We also integrate detection and segmentation capabilities into a unified MLLM framework, training it with a novel adaptive weighted loss to optimize performance.
  arXiv  Detail & Related papers  (2025-10-03T08:50:21Z)
• GRIL: Knowledge Graph Retrieval-Integrated Learning with Large Language Models [59.72897499248909]
  We propose a novel graph retriever trained end-to-end with Large Language Models (LLMs)<n>Within the extracted subgraph, structural knowledge and semantic features are encoded via soft tokens and the verbalized graph, respectively, which are infused into the LLM together.<n>Our approach consistently achieves state-of-the-art performance, validating the strength of joint graph-LLM optimization for complex reasoning tasks.
  arXiv  Detail & Related papers  (2025-09-20T02:38:00Z)
• A Graph Talks, But Who's Listening? Rethinking Evaluations for Graph-Language Models [11.808687414968388]
  Developments in Graph-Language Models (GLMs) aim to integrate the structural reasoning capabilities of Graph Neural Networks (GNNs) with the semantic understanding of Large Language Models (LLMs)<n>We demonstrate that current evaluation benchmarks for GLMs are insufficient to assess multimodal reasoning.<n>We introduce the CLEGR(Compositional Language-Graph Reasoning) benchmark, designed to evaluate multimodal reasoning at various complexity levels.
  arXiv  Detail & Related papers  (2025-08-28T09:20:47Z)
• Integrating Neural and Symbolic Components in a Model of Pragmatic Question-Answering [9.043409663314419]
  We propose a neuro-symbolic framework that enhances probabilistic cognitive models.<n>We examine various approaches to incorporating neural modules into the cognitive model.<n>We find that hybrid models can match or exceed the performance of traditional probabilistic models in predicting human answer patterns.
  arXiv  Detail & Related papers  (2025-06-02T09:34:37Z)
• Adaptive Graph of Thoughts: Test-Time Adaptive Reasoning Unifying Chain, Tree, and Graph Structures [0.0]
  We introduce Adaptive Graph of Thoughts (AGoT), a dynamic, graph-based inference framework.<n>AGoT enhances Large Language Models (LLMs) reasoning solely at test time.<n>We validate our approach on diverse benchmarks spanning multi-hop retrieval, scientific reasoning, and mathematical problem-solving.
  arXiv  Detail & Related papers  (2025-02-07T16:54:19Z)
• NeSyCoCo: A Neuro-Symbolic Concept Composer for Compositional Generalization [17.49136753589057]
  NeSyCoCo is a neuro-symbolic framework that generates symbolic representations and maps them to differentiable neural computations.<n>Our framework achieves state-of-the-art results on the ReaSCAN and CLEVR-CoGenT compositional generalization benchmarks.
  arXiv  Detail & Related papers  (2024-12-20T05:48:58Z)
• The Buffer Mechanism for Multi-Step Information Reasoning in Language Models [52.77133661679439]
  Investigating internal reasoning mechanisms of large language models can help us design better model architectures and training strategies. In this study, we constructed a symbolic dataset to investigate the mechanisms by which Transformer models employ vertical thinking strategy. We proposed a random matrix-based algorithm to enhance the model's reasoning ability, resulting in a 75% reduction in the training time required for the GPT-2 model.
  arXiv  Detail & Related papers  (2024-05-24T07:41:26Z)
• MIDGARD: Self-Consistency Using Minimum Description Length for Structured Commonsense Reasoning [4.928535836750263]
  We study the task of conducting structured reasoning as generating a reasoning graph from natural language input using large language models (LLMs) Previous approaches have explored various prompting schemes, yet they suffer from error propagation due to the autoregressive nature and single-pass-based decoding. We draw inspiration from self-consistency (SC), which involves sampling a diverse set of reasoning chains and taking the majority vote as the final answer.
  arXiv  Detail & Related papers  (2024-05-08T16:25:42Z)
• Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation [52.270712965271656]
  We propose a new model of contextual word representation, not from a neural perspective, but from a purely syntactic and probabilistic perspective. We find that the graph of our model resembles transformers, with correspondences between dependencies and self-attention. Experiments show that our model performs competitively to transformers on small to medium sized datasets.
  arXiv  Detail & Related papers  (2023-11-26T06:56:02Z)
• Discrete, compositional, and symbolic representations through attractor dynamics [51.20712945239422]
  We introduce a novel neural systems model that integrates attractor dynamics with symbolic representations to model cognitive processes akin to the probabilistic language of thought (PLoT) Our model segments the continuous representational space into discrete basins, with attractor states corresponding to symbolic sequences, that reflect the semanticity and compositionality characteristic of symbolic systems through unsupervised learning, rather than relying on pre-defined primitives. This approach establishes a unified framework that integrates both symbolic and sub-symbolic processing through neural dynamics, a neuroplausible substrate with proven expressivity in AI, offering a more comprehensive model that mirrors the complex duality of cognitive operations
  arXiv  Detail & Related papers  (2023-10-03T05:40:56Z)
• A Recursive Bateson-Inspired Model for the Generation of Semantic Formal Concepts from Spatial Sensory Data [77.34726150561087]
  This paper presents a new symbolic-only method for the generation of hierarchical concept structures from complex sensory data. The approach is based on Bateson's notion of difference as the key to the genesis of an idea or a concept. The model is able to produce fairly rich yet human-readable conceptual representations without training.
  arXiv  Detail & Related papers  (2023-07-16T15:59:13Z)
• Constructing Word-Context-Coupled Space Aligned with Associative Knowledge Relations for Interpretable Language Modeling [0.0]
  The black-box structure of the deep neural network in pre-trained language models seriously limits the interpretability of the language modeling process. A Word-Context-Coupled Space (W2CSpace) is proposed by introducing the alignment processing between uninterpretable neural representation and interpretable statistical logic. Our language model can achieve better performance and highly credible interpretable ability compared to related state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-19T09:26:02Z)
• Learning Disentangled Representations for Natural Language Definitions [0.0]
  We argue that recurrent syntactic and semantic regularities in textual data can be used to provide the models with both structural biases and generative factors. We leverage the semantic structures present in a representative and semantically dense category of sentence types, definitional sentences, for training a Variational Autoencoder to learn disentangled representations.
  arXiv  Detail & Related papers  (2022-09-22T14:31:55Z)
• GraphQ IR: Unifying Semantic Parsing of Graph Query Language with Intermediate Representation [91.27083732371453]
  We propose a unified intermediate representation (IR) for graph query languages, namely GraphQ IR. With the IR's natural-language-like representation that bridges the semantic gap and its formally defined syntax that maintains the graph structure, neural semantic parsing can more effectively convert user queries into GraphQ IR. Our approach can consistently achieve state-of-the-art performance on KQA Pro, Overnight and MetaQA.
  arXiv  Detail & Related papers  (2022-05-24T13:59:53Z)
• Hierarchical Memory Learning for Fine-Grained Scene Graph Generation [49.39355372599507]
  This paper proposes a novel Hierarchical Memory Learning (HML) framework to learn the model from simple to complex. After the autonomous partition of coarse and fine predicates, the model is first trained on the coarse predicates and then learns the fine predicates.
  arXiv  Detail & Related papers  (2022-03-14T08:01:14Z)
• Exploring End-to-End Differentiable Natural Logic Modeling [21.994060519995855]
  We explore end-to-end trained differentiable models that integrate natural logic with neural networks. The proposed model adapts module networks to model natural logic operations, which is enhanced with a memory component to model contextual information.
  arXiv  Detail & Related papers  (2020-11-08T18:18:15Z)
• Explaining and Improving Model Behavior with k Nearest Neighbor Representations [107.24850861390196]
  We propose using k nearest neighbor representations to identify training examples responsible for a model's predictions. We show that kNN representations are effective at uncovering learned spurious associations. Our results indicate that the kNN approach makes the finetuned model more robust to adversarial inputs.
  arXiv  Detail & Related papers  (2020-10-18T16:55:25Z)
• High-order Semantic Role Labeling [86.29371274587146]
  This paper introduces a high-order graph structure for the neural semantic role labeling model. It enables the model to explicitly consider not only the isolated predicate-argument pairs but also the interaction between the predicate-argument pairs. Experimental results on 7 languages of the CoNLL-2009 benchmark show that the high-order structural learning techniques are beneficial to the strong performing SRL models.
  arXiv  Detail & Related papers  (2020-10-09T15:33:54Z)
• Overestimation of Syntactic Representationin Neural Language Models [16.765097098482286]
  One popular method for determining a model's ability to induce syntactic structure trains a model on strings generated according to a template then tests the model's ability to distinguish such strings from superficially similar ones with different syntax. We illustrate a fundamental problem with this approach by reproducing positive results from a recent paper with two non-syntactic baseline language models.
  arXiv  Detail & Related papers  (2020-04-10T15:13:03Z)
• Embedding Graph Auto-Encoder for Graph Clustering [90.8576971748142]
  Graph auto-encoder (GAE) models are based on semi-supervised graph convolution networks (GCN) We design a specific GAE-based model for graph clustering to be consistent with the theory, namely Embedding Graph Auto-Encoder (EGAE) EGAE consists of one encoder and dual decoders.
  arXiv  Detail & Related papers  (2020-02-20T09:53:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.