Object-centric Cross-modal Feature Distillation for Event-based Object
Detection
- URL: http://arxiv.org/abs/2311.05494v1
- Date: Thu, 9 Nov 2023 16:33:08 GMT
- Title: Object-centric Cross-modal Feature Distillation for Event-based Object
Detection
- Authors: Lei Li, Alexander Liniger, Mario Millhaeusler, Vagia Tsiminaki,
Yuanyou Li, Dengxin Dai
- Abstract summary: RGB detectors still outperform event-based detectors due to sparsity of the event data and missing visual details.
We develop a novel knowledge distillation approach to shrink the performance gap between these two modalities.
We show that object-centric distillation allows to significantly improve the performance of the event-based student object detector.
- Score: 87.50272918262361
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Event cameras are gaining popularity due to their unique properties, such as
their low latency and high dynamic range. One task where these benefits can be
crucial is real-time object detection. However, RGB detectors still outperform
event-based detectors due to the sparsity of the event data and missing visual
details. In this paper, we develop a novel knowledge distillation approach to
shrink the performance gap between these two modalities. To this end, we
propose a cross-modality object detection distillation method that by design
can focus on regions where the knowledge distillation works best. We achieve
this by using an object-centric slot attention mechanism that can iteratively
decouple features maps into object-centric features and corresponding
pixel-features used for distillation. We evaluate our novel distillation
approach on a synthetic and a real event dataset with aligned grayscale images
as a teacher modality. We show that object-centric distillation allows to
significantly improve the performance of the event-based student object
detector, nearly halving the performance gap with respect to the teacher.
Related papers
- Towards Consistent Object Detection via LiDAR-Camera Synergy [17.665362927472973]
This paper introduces an end-to-end Consistency Object Detection (COD) algorithm framework.
It simultaneously obtains an object's position in both point clouds and images and establish their correlation.
To assess the accuracy of the object correlation between point clouds and images, this paper proposes a new evaluation metric, Consistency Precision.
arXiv Detail & Related papers (2024-05-02T13:04:26Z) - Innovative Horizons in Aerial Imagery: LSKNet Meets DiffusionDet for
Advanced Object Detection [55.2480439325792]
We present an in-depth evaluation of an object detection model that integrates the LSKNet backbone with the DiffusionDet head.
The proposed model achieves a mean average precision (MAP) of approximately 45.7%, which is a significant improvement.
This advancement underscores the effectiveness of the proposed modifications and sets a new benchmark in aerial image analysis.
arXiv Detail & Related papers (2023-11-21T19:49:13Z) - Efficient Object Detection in Optical Remote Sensing Imagery via
Attention-based Feature Distillation [29.821082433621868]
We propose Attention-based Feature Distillation (AFD) for object detection.
We introduce a multi-instance attention mechanism that effectively distinguishes between background and foreground elements.
AFD attains the performance of other state-of-the-art models while being efficient.
arXiv Detail & Related papers (2023-10-28T11:15:37Z) - Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
Unsupervised object discovery is promising due to its ability to discover objects in a generic manner.
We design a semantic-guided self-supervised learning model to extract high-level semantic features from images.
We introduce Principal Component Analysis (PCA) to localize object regions.
arXiv Detail & Related papers (2023-07-07T04:03:48Z) - AMD: Adaptive Masked Distillation for Object [8.668808292258706]
We propose a spatial-channel adaptive masked distillation (AMD) network for object detection.
We employ a simple and efficient module to allow the student network channel to be adaptive.
With the help of our proposed distillation method, the student networks report 41.3%, 42.4%, and 42.7% mAP scores.
arXiv Detail & Related papers (2023-01-31T10:32:13Z) - AGO-Net: Association-Guided 3D Point Cloud Object Detection Network [86.10213302724085]
We propose a novel 3D detection framework that associates intact features for objects via domain adaptation.
We achieve new state-of-the-art performance on the KITTI 3D detection benchmark in both accuracy and speed.
arXiv Detail & Related papers (2022-08-24T16:54:38Z) - Knowledge Distillation for Oriented Object Detection on Aerial Images [1.827510863075184]
We present a model compression method for rotated object detection on aerial images by knowledge distillation, namely KD-RNet.
The experimental result on a large-scale aerial object detection dataset (DOTA) demonstrates that the proposed KD-RNet model can achieve improved mean-average precision (mAP) with reduced number of parameters, at the same time, KD-RNet boost the performance on providing high quality detections with higher overlap with groundtruth annotations.
arXiv Detail & Related papers (2022-06-20T14:24:16Z) - Localization Distillation for Object Detection [134.12664548771534]
Previous knowledge distillation (KD) methods for object detection mostly focus on feature imitation instead of mimicking the classification logits.
We present a novel localization distillation (LD) method which can efficiently transfer the localization knowledge from the teacher to the student.
We show that logit mimicking can outperform feature imitation and the absence of localization distillation is a critical reason for why logit mimicking underperforms for years.
arXiv Detail & Related papers (2022-04-12T17:14:34Z) - Robust Region Feature Synthesizer for Zero-Shot Object Detection [87.79902339984142]
We build a novel zero-shot object detection framework that contains an Intra-class Semantic Diverging component and an Inter-class Structure Preserving component.
It is the first study to carry out zero-shot object detection in remote sensing imagery.
arXiv Detail & Related papers (2022-01-01T03:09:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.