Neuro-Symbolic Spatio-Temporal Reasoning
- URL: http://arxiv.org/abs/2211.15566v1
- Date: Mon, 28 Nov 2022 17:21:41 GMT
- Title: Neuro-Symbolic Spatio-Temporal Reasoning
- Authors: Jae Hee Lee, Michael Sioutis, Kyra Ahrens, Marjan Alirezaie, Matthias
Kerzel, Stefan Wermter
- Abstract summary: Spatio-temporal knowledge is required beyond interacting with the physical world.
Different attempts have been made to integrate this into AI systems.
We propose a synergy between logical reasoning and machine learning grounded on spatial and temporal knowledge.
- Score: 16.29507250221851
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Knowledge about space and time is necessary to solve problems in the physical
world: An AI agent situated in the physical world and interacting with objects
often needs to reason about positions of and relations between objects; and as
soon as the agent plans its actions to solve a task, it needs to consider the
temporal aspect (e.g., what actions to perform over time). Spatio-temporal
knowledge, however, is required beyond interacting with the physical world, and
is also often transferred to the abstract world of concepts through analogies
and metaphors (e.g., "a threat that is hanging over our heads"). As spatial and
temporal reasoning is ubiquitous, different attempts have been made to
integrate this into AI systems. In the area of knowledge representation,
spatial and temporal reasoning has been largely limited to modeling objects and
relations and developing reasoning methods to verify statements about objects
and relations. On the other hand, neural network researchers have tried to
teach models to learn spatial relations from data with limited reasoning
capabilities. Bridging the gap between these two approaches in a mutually
beneficial way could allow us to tackle many complex real-world problems, such
as natural language processing, visual question answering, and semantic image
segmentation. In this chapter, we view this integration problem from the
perspective of Neuro-Symbolic AI. Specifically, we propose a synergy between
logical reasoning and machine learning that will be grounded on spatial and
temporal knowledge. Describing some successful applications, remaining
challenges, and evaluation datasets pertaining to this direction is the main
topic of this contribution.
Related papers
- STAR: A Benchmark for Situated Reasoning in Real-World Videos [94.78038233351758]
This paper introduces a new benchmark that evaluates the situated reasoning ability via situation abstraction and logic-grounded question answering for real-world videos.
The dataset includes four types of questions, including interaction, sequence, prediction, and feasibility.
We propose a diagnostic neuro-symbolic model that can disentangle visual perception, situation abstraction, language understanding, and functional reasoning.
arXiv Detail & Related papers (2024-05-15T21:53:54Z) - Reconciling Spatial and Temporal Abstractions for Goal Representation [0.4813333335683418]
Goal representation affects the performance of Hierarchical Reinforcement Learning (HRL) algorithms.
Recent studies show that representations that preserve temporally abstract environment dynamics are successful in solving difficult problems.
We propose a novel three-layer HRL algorithm that introduces, at different levels of the hierarchy, both a spatial and a temporal goal abstraction.
arXiv Detail & Related papers (2024-01-18T10:33:30Z) - I-PHYRE: Interactive Physical Reasoning [25.02124862476204]
I-PHYRE is a framework that challenges agents to simultaneously exhibit intuitive physical reasoning, multi-step planning, and in-situ intervention.
We formulate four game splits to scrutinize agents' learning and generalization of essential principles of interactive physical reasoning.
arXiv Detail & Related papers (2023-12-04T19:01:19Z) - Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic
Systems [67.01132165581667]
We propose to enable high-level reasoning in AI systems by integrating cognitive architectures with external neuro-symbolic components.
We illustrate a hybrid framework centered on ACT-R and we discuss the role of generative models in recent and future applications.
arXiv Detail & Related papers (2023-11-13T21:20:17Z) - Causal Deep Learning [77.49632479298745]
Causality has the potential to transform the way we solve real-world problems.
But causality often requires crucial assumptions which cannot be tested in practice.
We propose a new way of thinking about causality -- we call this causal deep learning.
arXiv Detail & Related papers (2023-03-03T19:19:18Z) - Neural Fields in Visual Computing and Beyond [54.950885364735804]
Recent advances in machine learning have created increasing interest in solving visual computing problems using coordinate-based neural networks.
neural fields have seen successful application in the synthesis of 3D shapes and image, animation of human bodies, 3D reconstruction, and pose estimation.
This report provides context, mathematical grounding, and an extensive review of literature on neural fields.
arXiv Detail & Related papers (2021-11-22T18:57:51Z) - HALMA: Humanlike Abstraction Learning Meets Affordance in Rapid Problem
Solving [104.79156980475686]
Humans learn compositional and causal abstraction, ie, knowledge, in response to the structure of naturalistic tasks.
We argue there shall be three levels of generalization in how an agent represents its knowledge: perceptual, conceptual, and algorithmic.
This benchmark is centered around a novel task domain, HALMA, for visual concept development and rapid problem-solving.
arXiv Detail & Related papers (2021-02-22T20:37:01Z) - On the Binding Problem in Artificial Neural Networks [12.04468744445707]
We argue that the underlying cause for this shortcoming is their inability to dynamically and flexibly bind information.
We propose a unifying framework that revolves around forming meaningful entities from unstructured sensory inputs.
We believe that a compositional approach to AI, in terms of grounded symbol-like representations, is of fundamental importance for realizing human-level generalization.
arXiv Detail & Related papers (2020-12-09T18:02:49Z) - Neurosymbolic AI for Situated Language Understanding [13.249453757295083]
We argue that computational situated grounding provides a solution to some of these learning challenges.
Our model reincorporates some ideas of classic AI into a framework of neurosymbolic intelligence.
We discuss how situated grounding provides diverse data and multiple levels of modeling for a variety of AI learning challenges.
arXiv Detail & Related papers (2020-12-05T05:03:28Z) - Machine Common Sense [77.34726150561087]
Machine common sense remains a broad, potentially unbounded problem in artificial intelligence (AI)
This article deals with the aspects of modeling commonsense reasoning focusing on such domain as interpersonal interactions.
arXiv Detail & Related papers (2020-06-15T13:59:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.