Event Collapse in Contrast Maximization Frameworks

• URL: http://arxiv.org/abs/2207.04007v2
• Date: Mon, 11 Jul 2022 12:56:57 GMT
• Title: Event Collapse in Contrast Maximization Frameworks
• Authors: Shintaro Shiba, Yoshimitsu Aoki, Guillermo Gallego
• Abstract summary: ContrastMax (C) is a framework that provides state-of-the-art results on several event-based computer vision, tasks such as optical ego-motion or flow estimation. However, it may suffer from a problem called event collapse, which is an undesired solution where events are warped into too few pixels. Our work demonstrates event collapse in its simplest form and proposes collapse metrics by using first principles of space-time deformation.
• Score: 13.298845944779108
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Contrast maximization (CMax) is a framework that provides state-of-the-art results on several event-based computer vision tasks, such as ego-motion or optical flow estimation. However, it may suffer from a problem called event collapse, which is an undesired solution where events are warped into too few pixels. As prior works have largely ignored the issue or proposed workarounds, it is imperative to analyze this phenomenon in detail. Our work demonstrates event collapse in its simplest form and proposes collapse metrics by using first principles of space-time deformation based on differential geometry and physics. We experimentally show on publicly available datasets that the proposed metrics mitigate event collapse and do not harm well-posed warps. To the best of our knowledge, regularizers based on the proposed metrics are the only effective solution against event collapse in the experimental settings considered, compared with other methods. We hope that this work inspires further research to tackle more complex warp models.

Related papers

• Event-based Photometric Bundle Adjustment [12.504055397619727]
  Event-based Photometric Bundle Adjustment (EPBA) is the first event-only photometric bundle adjustment method. EPBA is effective in decreasing the photometric error (by up to 90%) Experiments on modern high-resolution event cameras show the applicability of EPBA to panoramic imaging.
  arXiv  Detail & Related papers  (2024-12-18T17:58:16Z)
• CMax-SLAM: Event-based Rotational-Motion Bundle Adjustment and SLAM System using Contrast Maximization [14.771885020122062]
  Event cameras are bio-inspired visual sensors that capture pixel-wise intensity changes and output asynchronous event streams. This paper considers the problem of rotational motion estimation using event cameras. Several event-based rotation estimation methods have been developed in the past decade, but their performance has not been evaluated.
  arXiv  Detail & Related papers  (2024-03-12T23:05:10Z)
• Improving Event Definition Following For Zero-Shot Event Detection [66.27883872707523]
  Existing approaches on zero-shot event detection usually train models on datasets annotated with known event types. We aim to improve zero-shot event detection by training models to better follow event definitions.
  arXiv  Detail & Related papers  (2024-03-05T01:46:50Z)
• From Simple to Complex: A Progressive Framework for Document-level Informative Argument Extraction [34.37013964529546]
  Event Argument Extraction (EAE) requires the model to extract arguments of multiple events from a single document. We propose a simple-to-complex progressive framework for document-level EAE. Our model outperforms SOTA by 1.4% in F1, indicating the proposed simple-to-complex framework is useful in the EAE task.
  arXiv  Detail & Related papers  (2023-10-25T04:38:02Z)
• On the Embedding Collapse when Scaling up Recommendation Models [53.66285358088788]
  We identify the embedding collapse phenomenon as the inhibition of scalability, wherein the embedding matrix tends to occupy a low-dimensional subspace. We propose a simple yet effective multi-embedding design incorporating embedding-set-specific interaction modules to learn embedding sets with large diversity.
  arXiv  Detail & Related papers  (2023-10-06T17:50:38Z)
• A Fast Geometric Regularizer to Mitigate Event Collapse in the Contrast Maximization Framework [13.298845944779108]
  We propose a novel, computationally efficient regularizer based on geometric principles to mitigate event collapse. Experiments show that the proposed regularizer achieves state-of-the-art accuracy results, while its reduced computational complexity makes it two to four times faster than previous approaches.
  arXiv  Detail & Related papers  (2022-12-14T17:22:48Z)
• Secrets of Event-Based Optical Flow [13.298845944779108]
  Event cameras respond to scene dynamics and offer advantages to estimate motion. We develop a principled method to extend the Contrast Maximization framework to estimate optical flow from events alone. Our method ranks first among unsupervised methods on the MVSEC benchmark, and is competitive on the DSEC benchmark.
  arXiv  Detail & Related papers  (2022-07-20T16:40:38Z)
• Reliable Shot Identification for Complex Event Detection via Visual-Semantic Embedding [72.9370352430965]
  We propose a visual-semantic guided loss method for event detection in videos. Motivated by curriculum learning, we introduce a negative elastic regularization term to start training the classifier with instances of high reliability. An alternative optimization algorithm is developed to solve the proposed challenging non-net regularization problem.
  arXiv  Detail & Related papers  (2021-10-12T11:46:56Z)
• CoSeg: Cognitively Inspired Unsupervised Generic Event Segmentation [118.18977078626776]
  We propose an end-to-end self-supervised learning framework for event segmentation/boundary detection. Our framework exploits a transformer-based feature reconstruction scheme to detect event boundary by reconstruction errors. The goal of our work is to segment generic events rather than localize some specific ones.
  arXiv  Detail & Related papers  (2021-09-30T14:40:32Z)
• Learning Monocular Dense Depth from Events [53.078665310545745]
  Event cameras produce brightness changes in the form of a stream of asynchronous events instead of intensity frames. Recent learning-based approaches have been applied to event-based data, such as monocular depth prediction. We propose a recurrent architecture to solve this task and show significant improvement over standard feed-forward methods.
  arXiv  Detail & Related papers  (2020-10-16T12:36:23Z)
• A Background-Agnostic Framework with Adversarial Training for Abnormal Event Detection in Video [120.18562044084678]
  Abnormal event detection in video is a complex computer vision problem that has attracted significant attention in recent years. We propose a background-agnostic framework that learns from training videos containing only normal events.
  arXiv  Detail & Related papers  (2020-08-27T18:39:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.