Temporal and Object Quantification Networks
- URL: http://arxiv.org/abs/2106.05891v1
- Date: Thu, 10 Jun 2021 16:18:21 GMT
- Title: Temporal and Object Quantification Networks
- Authors: Jiayuan Mao, Zhezheng Luo, Chuang Gan, Joshua B. Tenenbaum, Jiajun Wu,
Leslie Pack Kaelbling, Tomer D. Ullman
- Abstract summary: We present a new class of neuro-symbolic networks with a structural bias that enables them to learn to recognize complex relational-temporal events.
We demonstrate that TOQ-Nets can generalize from small amounts of data to scenarios containing more objects than were present during training and to temporal warpings of input sequences.
- Score: 95.64650820186706
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present Temporal and Object Quantification Networks (TOQ-Nets), a new
class of neuro-symbolic networks with a structural bias that enables them to
learn to recognize complex relational-temporal events. This is done by
including reasoning layers that implement finite-domain quantification over
objects and time. The structure allows them to generalize directly to input
instances with varying numbers of objects in temporal sequences of varying
lengths. We evaluate TOQ-Nets on input domains that require recognizing
event-types in terms of complex temporal relational patterns. We demonstrate
that TOQ-Nets can generalize from small amounts of data to scenarios containing
more objects than were present during training and to temporal warpings of
input sequences.
Related papers
- Large Language Models as Event Forecasters [10.32127659470566]
Key elements of human events are extracted as quadruples that consist of subject, relation, object, and timestamp.
These quadruples or quintuples, when organized within a specific domain, form a temporal knowledge graph (TKG)
arXiv Detail & Related papers (2024-06-15T04:09:31Z) - Temporal-aware Hierarchical Mask Classification for Video Semantic
Segmentation [62.275143240798236]
Video semantic segmentation dataset has limited categories per video.
Less than 10% of queries could be matched to receive meaningful gradient updates during VSS training.
Our method achieves state-of-the-art performance on the latest challenging VSS benchmark VSPW without bells and whistles.
arXiv Detail & Related papers (2023-09-14T20:31:06Z) - How neural networks learn to classify chaotic time series [77.34726150561087]
We study the inner workings of neural networks trained to classify regular-versus-chaotic time series.
We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
arXiv Detail & Related papers (2023-06-04T08:53:27Z) - It is all Connected: A New Graph Formulation for Spatio-Temporal
Forecasting [1.278093617645299]
We propose a framework for learning temporal and spatial dependencies using Graph Neural Network (GNN) networks.
GNNs represent every sample as its own node in a graph, rather than all measurements for a particular location as a single node.
The framework does not require measurements along the temporal dimension, meaning that it also facilitates irregular time series, different frequencies or missing data, without the need for data sampling imputation.
arXiv Detail & Related papers (2023-03-23T11:16:33Z) - Tracking Objects and Activities with Attention for Temporal Sentence
Grounding [51.416914256782505]
Temporal sentence (TSG) aims to localize the temporal segment which is semantically aligned with a natural language query in an untrimmed segment.
We propose a novel Temporal Sentence Tracking Network (TSTNet), which contains (A) a Cross-modal Targets Generator to generate multi-modal and search space, and (B) a Temporal Sentence Tracker to track multi-modal targets' behavior and to predict query-related segment.
arXiv Detail & Related papers (2023-02-21T16:42:52Z) - Semi-Structured Object Sequence Encoders [9.257633944317735]
We focus on the problem of developing a structure-aware input representation for semi-structured object sequences.
This type of data is often represented as a sequence of sets of key-value pairs over time.
We propose a two-part approach, which first considers each key independently and encodes a representation of its values over time.
arXiv Detail & Related papers (2023-01-03T09:19:41Z) - Novel Features for Time Series Analysis: A Complex Networks Approach [62.997667081978825]
Time series data are ubiquitous in several domains as climate, economics and health care.
Recent conceptual approach relies on time series mapping to complex networks.
Network analysis can be used to characterize different types of time series.
arXiv Detail & Related papers (2021-10-11T13:46:28Z) - Learnable Dynamic Temporal Pooling for Time Series Classification [22.931314501371805]
We present a dynamic temporal pooling (DTP) technique that reduces the temporal size of hidden representations by aggregating the features at the segment-level.
For the partition of a whole series into multiple segments, we utilize dynamic time warping (DTW) to align each time point in a temporal order with the prototypical features of the segments.
The DTP layer combined with a fully-connected layer helps to extract further discriminative features considering their temporal position within an input time series.
arXiv Detail & Related papers (2021-04-02T08:58:44Z) - Hierarchical Conditional Relation Networks for Multimodal Video Question
Answering [67.85579756590478]
Video QA adds at least two more layers of complexity - selecting relevant content for each channel in the context of a linguistic query.
Conditional Relation Network (CRN) takes as input a set of tensorial objects translating into a new set of objects that encode relations of the inputs.
CRN is then applied for Video QA in two forms, short-form where answers are reasoned solely from the visual content, and long-form where associated information, such as subtitles, is presented.
arXiv Detail & Related papers (2020-10-18T02:31:06Z) - Instance Explainable Temporal Network For Multivariate Timeseries [0.0]
We propose a novel network (IETNet) that identifies the important channels in the classification decision for each instance of inference.
IETNet is an end-to-end network that combines temporal feature extraction, variable selection, and joint variable interaction into a single learning framework.
arXiv Detail & Related papers (2020-05-26T20:55:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.