Neuro Symbolic Knowledge Reasoning for Procedural Video Question Answering

• URL: http://arxiv.org/abs/2503.14957v1
• Date: Wed, 19 Mar 2025 07:49:14 GMT
• Title: Neuro Symbolic Knowledge Reasoning for Procedural Video Question Answering
• Authors: Thanh-Son Nguyen, Hong Yang, Tzeh Yuan Neoh, Hao Zhang, Ee Yeo Keat, Basura Fernando,
• Abstract summary: This paper introduces a new video question-answering dataset that challenges models to leverage procedural knowledge for complex reasoning.<n>It requires recognizing visual entities, generating hypotheses, and performing contextual, causal, and counterfactual reasoning.
• Score: 26.013577822475856
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper introduces a new video question-answering (VQA) dataset that challenges models to leverage procedural knowledge for complex reasoning. It requires recognizing visual entities, generating hypotheses, and performing contextual, causal, and counterfactual reasoning. To address this, we propose neuro symbolic reasoning module that integrates neural networks and LLM-driven constrained reasoning over variables for interpretable answer generation. Results show that combining LLMs with structured knowledge reasoning with logic enhances procedural reasoning on the STAR benchmark and our dataset. Code and dataset at https://github.com/LUNAProject22/KML soon.

Related papers

• An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMs [8.861378619584093]
  Large language models (LLMs) have shown strong arithmetic reasoning capabilities when prompted with Chain-of-Thought prompts. We investigate neuron activation'' as a lens to provide a unified explanation to observations made by prior work.
  arXiv  Detail & Related papers  (2024-06-18T05:49:24Z)
• Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning [89.89857766491475]
  We propose a curriculum-based logical-aware instruction tuning framework, named LACT.<n>Specifically, we augment the arbitrary first-order logical queries via binary tree decomposition.<n> Experiments across widely used datasets demonstrate that LACT has substantial improvements(brings an average +5.5% MRR score) over advanced methods, achieving the new state-of-the-art.
  arXiv  Detail & Related papers  (2024-05-02T18:12:08Z)
• IID Relaxation by Logical Expressivity: A Research Agenda for Fitting Logics to Neurosymbolic Requirements [50.57072342894621]
  We discuss the benefits of exploiting known data dependencies and distribution constraints for Neurosymbolic use cases. This opens a new research agenda with general questions about Neurosymbolic background knowledge and the expressivity required of its logic.
  arXiv  Detail & Related papers  (2024-04-30T12:09:53Z)
• keqing: knowledge-based question answering is a nature chain-of-thought mentor of LLM [27.76205400533089]
  Large language models (LLMs) have exhibited remarkable performance on various natural language processing (NLP) tasks, especially for question answering. We present a novel framework to assist LLMs, such as ChatGPT, to retrieve question-related structured information on the knowledge graph. The experimental results on KBQA datasets show that Keqing can achieve competitive performance and illustrate the logic of answering each question.
  arXiv  Detail & Related papers  (2023-12-31T08:39:04Z)
• LOGICSEG: Parsing Visual Semantics with Neural Logic Learning and Reasoning [73.98142349171552]
  LOGICSEG is a holistic visual semantic that integrates neural inductive learning and logic reasoning with both rich data and symbolic knowledge. During fuzzy logic-based continuous relaxation, logical formulae are grounded onto data and neural computational graphs, hence enabling logic-induced network training. These designs together make LOGICSEG a general and compact neural-logic machine that is readily integrated into existing segmentation models.
  arXiv  Detail & Related papers  (2023-09-24T05:43:19Z)
• Search-in-the-Chain: Interactively Enhancing Large Language Models with Search for Knowledge-intensive Tasks [121.74957524305283]
  This paper proposes a novel framework named textbfSearch-in-the-Chain (SearChain) for the interaction between Information Retrieval (IR) and Large Language Model (LLM) Experiments show that SearChain outperforms state-of-the-art baselines on complex knowledge-intensive tasks.
  arXiv  Detail & Related papers  (2023-04-28T10:15:25Z)
• Rethinking Complex Queries on Knowledge Graphs with Neural Link Predictors [58.340159346749964]
  We propose a new neural-symbolic method to support end-to-end learning using complex queries with provable reasoning capability. We develop a new dataset containing ten new types of queries with features that have never been considered. Our method outperforms previous methods significantly in the new dataset and also surpasses previous methods in the existing dataset at the same time.
  arXiv  Detail & Related papers  (2023-04-14T11:35:35Z)
• Neural-Symbolic Integration for Interactive Learning and Conceptual Grounding [1.14219428942199]
  We propose neural-symbolic integration for abstract concept explanation and interactive learning. Interaction with the user confirms or rejects a revision of the neural model. The approach is illustrated using the Logic Network framework alongside Concept Activation Vectors and applied to a Conal Neural Network.
  arXiv  Detail & Related papers  (2021-12-22T11:24:48Z)
• Question Answering over Knowledge Bases by Leveraging Semantic Parsing and Neuro-Symbolic Reasoning [73.00049753292316]
  We propose a semantic parsing and reasoning-based Neuro-Symbolic Question Answering(NSQA) system. NSQA achieves state-of-the-art performance on QALD-9 and LC-QuAD 1.0.
  arXiv  Detail & Related papers  (2020-12-03T05:17:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.