TogetherNet: Bridging Image Restoration and Object Detection Together
via Dynamic Enhancement Learning
- URL: http://arxiv.org/abs/2209.01373v1
- Date: Sat, 3 Sep 2022 09:06:13 GMT
- Title: TogetherNet: Bridging Image Restoration and Object Detection Together
via Dynamic Enhancement Learning
- Authors: Yongzhen Wang, Xuefeng Yan, Kaiwen Zhang, Lina Gong, Haoran Xie, Fu
Lee Wang, Mingqiang Wei
- Abstract summary: Adverse weather conditions such as haze, rain, and snow often impair the quality of captured images.
We propose an effective yet unified detection paradigm that bridges image restoration and object detection.
We show that our TogetherNet outperforms the state-of-the-art detection approaches by a large margin both quantitatively and qualitatively.
- Score: 20.312198020027957
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Adverse weather conditions such as haze, rain, and snow often impair the
quality of captured images, causing detection networks trained on normal images
to generalize poorly in these scenarios. In this paper, we raise an intriguing
question - if the combination of image restoration and object detection, can
boost the performance of cutting-edge detectors in adverse weather conditions.
To answer it, we propose an effective yet unified detection paradigm that
bridges these two subtasks together via dynamic enhancement learning to discern
objects in adverse weather conditions, called TogetherNet. Different from
existing efforts that intuitively apply image dehazing/deraining as a
pre-processing step, TogetherNet considers a multi-task joint learning problem.
Following the joint learning scheme, clean features produced by the restoration
network can be shared to learn better object detection in the detection
network, thus helping TogetherNet enhance the detection capacity in adverse
weather conditions. Besides the joint learning architecture, we design a new
Dynamic Transformer Feature Enhancement module to improve the feature
extraction and representation capabilities of TogetherNet. Extensive
experiments on both synthetic and real-world datasets demonstrate that our
TogetherNet outperforms the state-of-the-art detection approaches by a large
margin both quantitatively and qualitatively. Source code is available at
https://github.com/yz-wang/TogetherNet.
Related papers
- D-YOLO a robust framework for object detection in adverse weather conditions [0.0]
Adverse weather conditions including haze, snow and rain lead to decline in image qualities, which often causes a decline in performance for deep-learning based detection networks.
To better integrate image restoration and object detection tasks, we designed a double-route network with an attention feature fusion module.
We also proposed a subnetwork to provide haze-free features to the detection network. Specifically, our D-YOLO improves the performance of the detection network by minimizing the distance between the clear feature extraction subnetwork and detection network.
arXiv Detail & Related papers (2024-03-14T09:57:15Z) - Transformer based Multitask Learning for Image Captioning and Object
Detection [13.340784876489927]
This work introduces a novel multitask learning framework that combines image captioning and object detection into a joint model.
We propose TICOD, Transformer-based Image Captioning and Object detection model for jointly training both tasks.
Our model outperforms the baselines from image captioning literature by achieving a 3.65% improvement in BERTScore.
arXiv Detail & Related papers (2024-03-10T19:31:13Z) - FriendNet: Detection-Friendly Dehazing Network [24.372610892854283]
We propose an effective architecture that bridges image dehazing and object detection together via guidance information and task-driven learning.
FriendNet aims to deliver both high-quality perception and high detection capacity.
arXiv Detail & Related papers (2024-03-07T12:19:04Z) - Joint Perceptual Learning for Enhancement and Object Detection in
Underwater Scenarios [41.34564703212461]
We propose a bilevel optimization formulation for jointly learning underwater object detection and image enhancement.
Our method outputs visually favoring images and higher detection accuracy.
arXiv Detail & Related papers (2023-07-07T11:54:06Z) - An Interactively Reinforced Paradigm for Joint Infrared-Visible Image
Fusion and Saliency Object Detection [59.02821429555375]
This research focuses on the discovery and localization of hidden objects in the wild and serves unmanned systems.
Through empirical analysis, infrared and visible image fusion (IVIF) enables hard-to-find objects apparent.
multimodal salient object detection (SOD) accurately delineates the precise spatial location of objects within the picture.
arXiv Detail & Related papers (2023-05-17T06:48:35Z) - Adversarially-Aware Robust Object Detector [85.10894272034135]
We propose a Robust Detector (RobustDet) based on adversarially-aware convolution to disentangle gradients for model learning on clean and adversarial images.
Our model effectively disentangles gradients and significantly enhances the detection robustness with maintaining the detection ability on clean images.
arXiv Detail & Related papers (2022-07-13T13:59:59Z) - Multi-scale Attentive Image De-raining Networks via Neural Architecture
Search [23.53770663034919]
We develop a high-performance multi-scale attentive neural architecture search (MANAS) framework for image deraining.
The proposed method formulates a new multi-scale attention search space with multiple flexible modules that are favorite to the image de-raining task.
The internal multiscale attentive architecture of the de-raining network is searched automatically through a gradient-based search algorithm.
arXiv Detail & Related papers (2022-07-02T03:47:13Z) - High-resolution Iterative Feedback Network for Camouflaged Object
Detection [128.893782016078]
Spotting camouflaged objects that are visually assimilated into the background is tricky for object detection algorithms.
We aim to extract the high-resolution texture details to avoid the detail degradation that causes blurred vision in edges and boundaries.
We introduce a novel HitNet to refine the low-resolution representations by high-resolution features in an iterative feedback manner.
arXiv Detail & Related papers (2022-03-22T11:20:21Z) - Correlation-Aware Deep Tracking [83.51092789908677]
We propose a novel target-dependent feature network inspired by the self-/cross-attention scheme.
Our network deeply embeds cross-image feature correlation in multiple layers of the feature network.
Our model can be flexibly pre-trained on abundant unpaired images, leading to notably faster convergence than the existing methods.
arXiv Detail & Related papers (2022-03-03T11:53:54Z) - Online-updated High-order Collaborative Networks for Single Image
Deraining [51.22694467126883]
Single image deraining is an important task for some downstream artificial intelligence applications such as video surveillance and self-driving systems.
We propose a high-order collaborative network with multi-scale compact constraints and a bidirectional scale-content similarity mining module.
Our proposed method performs favorably against eleven state-of-the-art methods on five public synthetic and one real-world dataset.
arXiv Detail & Related papers (2022-02-14T09:09:08Z) - Learning Deep Context-Sensitive Decomposition for Low-Light Image
Enhancement [58.72667941107544]
A typical framework is to simultaneously estimate the illumination and reflectance, but they disregard the scene-level contextual information encapsulated in feature spaces.
We develop a new context-sensitive decomposition network architecture to exploit the scene-level contextual dependencies on spatial scales.
We develop a lightweight CSDNet (named LiteCSDNet) by reducing the number of channels.
arXiv Detail & Related papers (2021-12-09T06:25:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.