0-MMS: Zero-Shot Multi-Motion Segmentation With A Monocular Event Camera

• URL: http://arxiv.org/abs/2006.06158v2
• Date: Sat, 7 Nov 2020 01:58:31 GMT
• Title: 0-MMS: Zero-Shot Multi-Motion Segmentation With A Monocular Event Camera
• Authors: Chethan M. Parameshwara, Nitin J. Sanket, Chahat Deep Singh, Cornelia Ferm\"uller, and Yiannis Aloimonos
• Abstract summary: We present an approach for monocular multi-motion segmentation, which combines bottom-up feature tracking and top-down motion compensation into a unified pipeline. Using the events within a time-interval, our method segments the scene into multiple motions by splitting and merging. The approach was successfully evaluated on both challenging real-world and synthetic scenarios from the EV-IMO, EED, and MOD datasets.
• Score: 13.39518293550118
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Segmentation of moving objects in dynamic scenes is a key process in scene understanding for navigation tasks. Classical cameras suffer from motion blur in such scenarios rendering them effete. On the contrary, event cameras, because of their high temporal resolution and lack of motion blur, are tailor-made for this problem. We present an approach for monocular multi-motion segmentation, which combines bottom-up feature tracking and top-down motion compensation into a unified pipeline, which is the first of its kind to our knowledge. Using the events within a time-interval, our method segments the scene into multiple motions by splitting and merging. We further speed up our method by using the concept of motion propagation and cluster keyslices. The approach was successfully evaluated on both challenging real-world and synthetic scenarios from the EV-IMO, EED, and MOD datasets and outperformed the state-of-the-art detection rate by 12\%, achieving a new state-of-the-art average detection rate of 81.06%, 94.2% and 82.35% on the aforementioned datasets. To enable further research and systematic evaluation of multi-motion segmentation, we present and open-source a new dataset/benchmark called MOD++, which includes challenging sequences and extensive data stratification in-terms of camera and object motion, velocity magnitudes, direction, and rotational speeds.

Related papers

• DATAP-SfM: Dynamic-Aware Tracking Any Point for Robust Structure from Motion in the Wild [85.03973683867797]
  This paper proposes a concise, elegant, and robust pipeline to estimate smooth camera trajectories and obtain dense point clouds for casual videos in the wild. We show that the proposed method achieves state-of-the-art performance in terms of camera pose estimation even in complex dynamic challenge scenes.
  arXiv  Detail & Related papers  (2024-11-20T13:01:16Z)
• Motion Segmentation for Neuromorphic Aerial Surveillance [42.04157319642197]
  Event cameras offer superior temporal resolution, superior dynamic range, and minimal power requirements. Unlike traditional frame-based sensors that capture redundant information at fixed intervals, event cameras asynchronously record pixel-level brightness changes. We introduce a novel motion segmentation method that leverages self-supervised vision transformers on both event data and optical flow information.
  arXiv  Detail & Related papers  (2024-05-24T04:36:13Z)
• Out of the Room: Generalizing Event-Based Dynamic Motion Segmentation for Complex Scenes [10.936350433952668]
  Rapid and reliable identification of dynamic scene parts, also known as motion segmentation, is a key challenge for mobile sensors. Event cameras have the potential to overcome these limitations, but corresponding methods have only been demonstrated in smaller-scale indoor environments. This work presents an event-based method for class-agnostic motion segmentation that can successfully be deployed across complex large-scale outdoor environments too.
  arXiv  Detail & Related papers  (2024-03-07T14:59:34Z)
• Event-Free Moving Object Segmentation from Moving Ego Vehicle [88.33470650615162]
  Moving object segmentation (MOS) in dynamic scenes is an important, challenging, but under-explored research topic for autonomous driving. Most segmentation methods leverage motion cues obtained from optical flow maps. We propose to exploit event cameras for better video understanding, which provide rich motion cues without relying on optical flow.
  arXiv  Detail & Related papers  (2023-04-28T23:43:10Z)
• Event-based Simultaneous Localization and Mapping: A Comprehensive Survey [52.73728442921428]
  Review of event-based vSLAM algorithms that exploit the benefits of asynchronous and irregular event streams for localization and mapping tasks. Paper categorizes event-based vSLAM methods into four main categories: feature-based, direct, motion-compensation, and deep learning methods.
  arXiv  Detail & Related papers  (2023-04-19T16:21:14Z)
• Implicit Motion Handling for Video Camouflaged Object Detection [60.98467179649398]
  We propose a new video camouflaged object detection (VCOD) framework. It can exploit both short-term and long-term temporal consistency to detect camouflaged objects from video frames.
  arXiv  Detail & Related papers  (2022-03-14T17:55:41Z)
• Event-based Motion Segmentation by Cascaded Two-Level Multi-Model Fitting [44.97191206895915]
  We present a cascaded two-level multi-model fitting method for identifying independently moving objects with a monocular event camera. Experiments demonstrate the effectiveness and versatility of our method in real-world scenes with different motion patterns and an unknown number of moving objects.
  arXiv  Detail & Related papers  (2021-11-05T12:59:41Z)
• Learning to Segment Rigid Motions from Two Frames [72.14906744113125]
  We propose a modular network, motivated by a geometric analysis of what independent object motions can be recovered from an egomotion field. It takes two consecutive frames as input and predicts segmentation masks for the background and multiple rigidly moving objects, which are then parameterized by 3D rigid transformations. Our method achieves state-of-the-art performance for rigid motion segmentation on KITTI and Sintel.
  arXiv  Detail & Related papers  (2021-01-11T04:20:30Z)
• Event-based Motion Segmentation with Spatio-Temporal Graph Cuts [51.17064599766138]
  We have developed a method to identify independently objects acquired with an event-based camera. The method performs on par or better than the state of the art without having to predetermine the number of expected moving objects.
  arXiv  Detail & Related papers  (2020-12-16T04:06:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.