DynamicGlue: Epipolar and Time-Informed Data Association in Dynamic Environments using Graph Neural Networks

• URL: http://arxiv.org/abs/2403.11370v3
• Date: Mon, 1 Jul 2024 09:04:31 GMT
• Title: DynamicGlue: Epipolar and Time-Informed Data Association in Dynamic Environments using Graph Neural Networks
• Authors: Theresa Huber, Simon Schaefer, Stefan Leutenegger,
• Abstract summary: We propose a graph neural network-based sparse feature matching network to perform robust matching under challenging conditions. We employ a similar scheme of attentional aggregation over graph edges to enhance keypoint representations as state-of-the-art feature-matching networks. A series of experiments show the superior performance of our network as it excludes keypoints on moving objects compared to state-of-the-art feature matching networks.
• Score: 13.42760841894735
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The assumption of a static environment is common in many geometric computer vision tasks like SLAM but limits their applicability in highly dynamic scenes. Since these tasks rely on identifying point correspondences between input images within the static part of the environment, we propose a graph neural network-based sparse feature matching network designed to perform robust matching under challenging conditions while excluding keypoints on moving objects. We employ a similar scheme of attentional aggregation over graph edges to enhance keypoint representations as state-of-the-art feature-matching networks but augment the graph with epipolar and temporal information and vastly reduce the number of graph edges. Furthermore, we introduce a self-supervised training scheme to extract pseudo labels for image pairs in dynamic environments from exclusively unprocessed visual-inertial data. A series of experiments show the superior performance of our network as it excludes keypoints on moving objects compared to state-of-the-art feature matching networks while still achieving similar results regarding conventional matching metrics. When integrated into a SLAM system, our network significantly improves performance, especially in highly dynamic scenes.

Related papers

• Multi-Task Edge Prediction in Temporally-Dynamic Video Graphs [16.121140184388786]
  We propose MTD-GNN, a graph network for predicting temporally-dynamic edges for multiple types of relations. We show that modeling multiple relations in our temporal-dynamic graph network can be mutually beneficial.
  arXiv  Detail & Related papers  (2022-12-06T10:41:00Z)
• Graph Neural Networks for Multi-Robot Active Information Acquisition [15.900385823366117]
  A team of mobile robots, communicating through an underlying graph, estimates a hidden state expressing a phenomenon of interest. Existing approaches are either not scalable, unable to handle dynamic phenomena or not robust to changes in the communication graph. We propose an Information-aware Graph Block Network (I-GBNet) that aggregates information over the graph representation and provides sequential-decision making in a distributed manner.
  arXiv  Detail & Related papers  (2022-09-24T21:45:06Z)
• Dynamic Graph Message Passing Networks for Visual Recognition [112.49513303433606]
  Modelling long-range dependencies is critical for scene understanding tasks in computer vision. A fully-connected graph is beneficial for such modelling, but its computational overhead is prohibitive. We propose a dynamic graph message passing network, that significantly reduces the computational complexity.
  arXiv  Detail & Related papers  (2022-09-20T14:41:37Z)
• Time-aware Dynamic Graph Embedding for Asynchronous Structural Evolution [60.695162101159134]
  Existing works merely view a dynamic graph as a sequence of changes. We formulate dynamic graphs as temporal edge sequences associated with joining time of. vertex and timespan of edges. A time-aware Transformer is proposed to embed. vertex' dynamic connections and ToEs into the learned. vertex representations.
  arXiv  Detail & Related papers  (2022-07-01T15:32:56Z)
• Efficient Dynamic Graph Representation Learning at Scale [66.62859857734104]
  We propose Efficient Dynamic Graph lEarning (EDGE), which selectively expresses certain temporal dependency via training loss to improve the parallelism in computations. We show that EDGE can scale to dynamic graphs with millions of nodes and hundreds of millions of temporal events and achieve new state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2021-12-14T22:24:53Z)
• Temporal Graph Network Embedding with Causal Anonymous Walks Representations [54.05212871508062]
  We propose a novel approach for dynamic network representation learning based on Temporal Graph Network. For evaluation, we provide a benchmark pipeline for the evaluation of temporal network embeddings. We show the applicability and superior performance of our model in the real-world downstream graph machine learning task provided by one of the top European banks.
  arXiv  Detail & Related papers  (2021-08-19T15:39:52Z)
• RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks [53.15260967235835]
  We propose a novel framework that refines the output of such methods by utilizing a graph-based representation of instance masks. We train deep networks capable of sampling smart perturbations to the segmentations, and a graph neural network, which can encode relations between objects, to evaluate the segmentations. We demonstrate an application that uses uncertainty estimates generated by our method to guide a manipulator, leading to efficient understanding of cluttered scenes.
  arXiv  Detail & Related papers  (2021-06-29T20:29:29Z)
• Mutually exciting point process graphs for modelling dynamic networks [0.0]
  A new class of models for dynamic networks is proposed, called mutually exciting point process graphs (MEG) MEG is a scalable network-wide statistical model for point processes with dyadic marks, which can be used for anomaly detection. The model is tested on simulated graphs and real world computer network datasets, demonstrating excellent performance.
  arXiv  Detail & Related papers  (2021-02-11T10:14:55Z)
• Structural Temporal Graph Neural Networks for Anomaly Detection in Dynamic Graphs [54.13919050090926]
  We propose an end-to-end structural temporal Graph Neural Network model for detecting anomalous edges in dynamic graphs. In particular, we first extract the $h$-hop enclosing subgraph centered on the target edge and propose the node labeling function to identify the role of each node in the subgraph. Based on the extracted features, we utilize Gated recurrent units (GRUs) to capture the temporal information for anomaly detection.
  arXiv  Detail & Related papers  (2020-05-15T09:17:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.