Asynchronous Events-based Panoptic Segmentation using Graph Mixer Neural Network

• URL: http://arxiv.org/abs/2305.03640v1
• Date: Fri, 5 May 2023 15:56:46 GMT
• Title: Asynchronous Events-based Panoptic Segmentation using Graph Mixer Neural Network
• Authors: Sanket Kachole, Yusra Alkendi, Fariborz Baghaei Naeini, Dimitrios Makris, Yahya Zweiri
• Abstract summary: We propose the Graph Neural Network that includes a novel collaborative contextual mixing, applied to 3D event graphs formed on events. We evaluate the effectiveness of our proposed method on the Event-basedtemporal (ESD) dataset. Results show that our proposed approach outperforms state-of-the-art methods in terms of mean intersection over the union and pixel accuracy.
• Score: 3.867356784754811
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the context of robotic grasping, object segmentation encounters several difficulties when faced with dynamic conditions such as real-time operation, occlusion, low lighting, motion blur, and object size variability. In response to these challenges, we propose the Graph Mixer Neural Network that includes a novel collaborative contextual mixing layer, applied to 3D event graphs formed on asynchronous events. The proposed layer is designed to spread spatiotemporal correlation within an event graph at four nearest neighbor levels parallelly. We evaluate the effectiveness of our proposed method on the Event-based Segmentation (ESD) Dataset, which includes five unique image degradation challenges, including occlusion, blur, brightness, trajectory, scale variance, and segmentation of known and unknown objects. The results show that our proposed approach outperforms state-of-the-art methods in terms of mean intersection over the union and pixel accuracy. Code available at: https://github.com/sanket0707/GNN-Mixer.git

Related papers

• Multi-Modal Graph Convolutional Network with Sinusoidal Encoding for Robust Human Action Segmentation [10.122882293302787]
  temporal segmentation of human actions is critical for intelligent robots in collaborative settings.<n>We propose a Multi-Modal Graph Convolutional Network (MMGCN) that integrates low-frame-rate (e.g., 1 fps) visual data with high-frame-rate (e.g., 30 fps) motion data.<n>Our approach outperforms state-of-the-art methods, especially in action segmentation accuracy.
  arXiv  Detail & Related papers  (2025-07-01T13:55:57Z)
• TESGNN: Temporal Equivariant Scene Graph Neural Networks for Efficient and Robust Multi-View 3D Scene Understanding [8.32401190051443]
  We present the first implementation of an Equivariant Scene Graph Neural Network (ESGNN) to generate semantic scene graphs from 3D point clouds. Our combined architecture, termed the Temporal Equivariant Scene Graph Neural Network (TESGNN), not only surpasses existing state-of-the-art methods in scene estimation accuracy but also achieves faster convergence.
  arXiv  Detail & Related papers  (2024-11-15T15:39:04Z)
• GeoMix: Towards Geometry-Aware Data Augmentation [76.09914619612812]
  Mixup has shown considerable success in mitigating the challenges posed by limited labeled data in image classification. We propose Geometric Mixup (GeoMix), a simple and interpretable Mixup approach leveraging in-place graph editing.
  arXiv  Detail & Related papers  (2024-07-15T12:58:04Z)
• Neuromorphic Vision-based Motion Segmentation with Graph Transformer Neural Network [4.386534439007928]
  We propose a novel event-based motion segmentation algorithm using a Graph Transformer Neural Network, dubbed GTNN. Our proposed algorithm processes event streams as 3D graphs by a series nonlinear transformations to unveil local and global correlations between events. We show that GTNN outperforms state-of-the-art methods in the presence of dynamic background variations, motion patterns, and multiple dynamic objects with varying sizes and velocities.
  arXiv  Detail & Related papers  (2024-04-16T22:44:29Z)
• Graph Transformer GANs with Graph Masked Modeling for Architectural Layout Generation [153.92387500677023]
  We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node relations. The proposed graph Transformer encoder combines graph convolutions and self-attentions in a Transformer to model both local and global interactions. We also propose a novel self-guided pre-training method for graph representation learning.
  arXiv  Detail & Related papers  (2024-01-15T14:36:38Z)
• Implicit Event-RGBD Neural SLAM [54.74363487009845]
  Implicit neural SLAM has achieved remarkable progress recently. Existing methods face significant challenges in non-ideal scenarios. We propose EN-SLAM, the first event-RGBD implicit neural SLAM framework.
  arXiv  Detail & Related papers  (2023-11-18T08:48:58Z)
• Graph-based Asynchronous Event Processing for Rapid Object Recognition [59.112755601918074]
  Event cameras capture asynchronous events stream in which each event encodes pixel location, trigger time, and the polarity of the brightness changes. We introduce a novel graph-based framework for event cameras, namely SlideGCN. Our approach can efficiently process data event-by-event, unlock the low latency nature of events data while still maintaining the graph's structure internally.
  arXiv  Detail & Related papers  (2023-08-28T08:59:57Z)
• Bimodal SegNet: Instance Segmentation Fusing Events and RGB Frames for Robotic Grasping [4.191965713559235]
  We propose a Deep Learning network that fuses two types of visual signals, event-based data and RGB frame data. The Bimodal SegNet network has two distinct encoders, one for each signal input and a spatial pyramidal pooling with atrous convolutions. The evaluation results show a 6-10% improvement over state-of-the-art methods in terms of mean intersection over the union and pixel accuracy.
  arXiv  Detail & Related papers  (2023-03-20T16:09:25Z)
• AEGNN: Asynchronous Event-based Graph Neural Networks [54.528926463775946]
  Event-based Graph Neural Networks generalize standard GNNs to process events as "evolving"-temporal graphs. AEGNNs are easily trained on synchronous inputs and can be converted to efficient, "asynchronous" networks at test time.
  arXiv  Detail & Related papers  (2022-03-31T16:21:12Z)
• RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks [53.15260967235835]
  We propose a novel framework that refines the output of such methods by utilizing a graph-based representation of instance masks. We train deep networks capable of sampling smart perturbations to the segmentations, and a graph neural network, which can encode relations between objects, to evaluate the segmentations. We demonstrate an application that uses uncertainty estimates generated by our method to guide a manipulator, leading to efficient understanding of cluttered scenes.
  arXiv  Detail & Related papers  (2021-06-29T20:29:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.