Object-centric Cross-modal Feature Distillation for Event-based Object Detection

• URL: http://arxiv.org/abs/2311.05494v1
• Date: Thu, 9 Nov 2023 16:33:08 GMT
• Title: Object-centric Cross-modal Feature Distillation for Event-based Object Detection
• Authors: Lei Li, Alexander Liniger, Mario Millhaeusler, Vagia Tsiminaki, Yuanyou Li, Dengxin Dai
• Abstract summary: RGB detectors still outperform event-based detectors due to sparsity of the event data and missing visual details. We develop a novel knowledge distillation approach to shrink the performance gap between these two modalities. We show that object-centric distillation allows to significantly improve the performance of the event-based student object detector.
• Score: 87.50272918262361
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Event cameras are gaining popularity due to their unique properties, such as their low latency and high dynamic range. One task where these benefits can be crucial is real-time object detection. However, RGB detectors still outperform event-based detectors due to the sparsity of the event data and missing visual details. In this paper, we develop a novel knowledge distillation approach to shrink the performance gap between these two modalities. To this end, we propose a cross-modality object detection distillation method that by design can focus on regions where the knowledge distillation works best. We achieve this by using an object-centric slot attention mechanism that can iteratively decouple features maps into object-centric features and corresponding pixel-features used for distillation. We evaluate our novel distillation approach on a synthetic and a real event dataset with aligned grayscale images as a teacher modality. We show that object-centric distillation allows to significantly improve the performance of the event-based student object detector, nearly halving the performance gap with respect to the teacher.

Related papers

• SurANet: Surrounding-Aware Network for Concealed Object Detection via Highly-Efficient Interactive Contrastive Learning Strategy [55.570183323356964]
  We propose a novel Surrounding-Aware Network, namely SurANet, for concealed object detection. We enhance the semantics of feature maps using differential fusion of surrounding features to highlight concealed objects. Next, a Surrounding-Aware Contrastive Loss is applied to identify the concealed object via learning surrounding feature maps contrastively.
  arXiv  Detail & Related papers  (2024-10-09T13:02:50Z)
• Towards Consistent Object Detection via LiDAR-Camera Synergy [17.665362927472973]
  There is no existing model capable of detecting an object's position in both point clouds and images. This paper introduces an end-to-end Consistency Object Detection (COD) algorithm framework. To assess the accuracy of the object correlation between point clouds and images, this paper proposes a new evaluation metric, Consistency Precision.
  arXiv  Detail & Related papers  (2024-05-02T13:04:26Z)
• Innovative Horizons in Aerial Imagery: LSKNet Meets DiffusionDet for Advanced Object Detection [55.2480439325792]
  We present an in-depth evaluation of an object detection model that integrates the LSKNet backbone with the DiffusionDet head. The proposed model achieves a mean average precision (MAP) of approximately 45.7%, which is a significant improvement. This advancement underscores the effectiveness of the proposed modifications and sets a new benchmark in aerial image analysis.
  arXiv  Detail & Related papers  (2023-11-21T19:49:13Z)
• Efficient Object Detection in Optical Remote Sensing Imagery via Attention-based Feature Distillation [29.821082433621868]
  We propose Attention-based Feature Distillation (AFD) for object detection. We introduce a multi-instance attention mechanism that effectively distinguishes between background and foreground elements. AFD attains the performance of other state-of-the-art models while being efficient.
  arXiv  Detail & Related papers  (2023-10-28T11:15:37Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• AMD: Adaptive Masked Distillation for Object [8.668808292258706]
  We propose a spatial-channel adaptive masked distillation (AMD) network for object detection. We employ a simple and efficient module to allow the student network channel to be adaptive. With the help of our proposed distillation method, the student networks report 41.3%, 42.4%, and 42.7% mAP scores.
  arXiv  Detail & Related papers  (2023-01-31T10:32:13Z)
• AGO-Net: Association-Guided 3D Point Cloud Object Detection Network [86.10213302724085]
  We propose a novel 3D detection framework that associates intact features for objects via domain adaptation. We achieve new state-of-the-art performance on the KITTI 3D detection benchmark in both accuracy and speed.
  arXiv  Detail & Related papers  (2022-08-24T16:54:38Z)
• Knowledge Distillation for Oriented Object Detection on Aerial Images [1.827510863075184]
  We present a model compression method for rotated object detection on aerial images by knowledge distillation, namely KD-RNet. The experimental result on a large-scale aerial object detection dataset (DOTA) demonstrates that the proposed KD-RNet model can achieve improved mean-average precision (mAP) with reduced number of parameters, at the same time, KD-RNet boost the performance on providing high quality detections with higher overlap with groundtruth annotations.
  arXiv  Detail & Related papers  (2022-06-20T14:24:16Z)
• Robust Region Feature Synthesizer for Zero-Shot Object Detection [87.79902339984142]
  We build a novel zero-shot object detection framework that contains an Intra-class Semantic Diverging component and an Inter-class Structure Preserving component. It is the first study to carry out zero-shot object detection in remote sensing imagery.
  arXiv  Detail & Related papers  (2022-01-01T03:09:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.