Convolution kernel adaptation to calibrated fisheye

• URL: http://arxiv.org/abs/2402.01456v1
• Date: Fri, 2 Feb 2024 14:44:50 GMT
• Title: Convolution kernel adaptation to calibrated fisheye
• Authors: Bruno Berenguel-Baeta, Maria Santos-Villafranca, Jesus Bermudez-Cameo, Alejandro Perez-Yus, Jose J. Guerrero
• Abstract summary: Convolution kernels are the basic structural component of convolutional neural networks (CNNs) We propose a method that leverages the calibration of cameras to deform the convolution kernel accordingly and adapt to the distortion. We show how, with just a brief fine-tuning stage in a small dataset, we improve the performance of the network for the calibrated fisheye.
• Score: 45.90423821963144
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Convolution kernels are the basic structural component of convolutional neural networks (CNNs). In the last years there has been a growing interest in fisheye cameras for many applications. However, the radially symmetric projection model of these cameras produces high distortions that affect the performance of CNNs, especially when the field of view is very large. In this work, we tackle this problem by proposing a method that leverages the calibration of cameras to deform the convolution kernel accordingly and adapt to the distortion. That way, the receptive field of the convolution is similar to standard convolutions in perspective images, allowing us to take advantage of pre-trained networks in large perspective datasets. We show how, with just a brief fine-tuning stage in a small dataset, we improve the performance of the network for the calibrated fisheye with respect to standard convolutions in depth estimation and semantic segmentation.

Related papers

• Deformable Convolution Based Road Scene Semantic Segmentation of Fisheye Images in Autonomous Driving [4.720434481945155]
  This study investigates the effectiveness of modern Deformable Convolutional Neural Networks (DCNNs) for semantic segmentation tasks. Our experiments focus on segmenting the WoodScape fisheye image dataset into ten distinct classes, assessing the Deformable Networks' ability to capture intricate spatial relationships. The significant improvement in mIoU score resulting from integrating Deformable CNNs demonstrates their effectiveness in handling the geometric distortions present in fisheye imagery.
  arXiv  Detail & Related papers  (2024-07-23T17:02:24Z)
• Adapting CNNs for Fisheye Cameras without Retraining [3.683202928838613]
  In many applications it is beneficial to use non conventional cameras, such as fisheye cameras, that have a larger field of view. The issue arises that these large-FOV images can't be rectified to a perspective projection without significant cropping of the original image. We propose Rectified Convolutions; a new approach for adapting pre-trained convolutional networks to operate with new non-perspective images.
  arXiv  Detail & Related papers  (2024-04-12T01:36:00Z)
• Sorted Convolutional Network for Achieving Continuous Rotational Invariance [56.42518353373004]
  We propose a Sorting Convolution (SC) inspired by some hand-crafted features of texture images. SC achieves continuous rotational invariance without requiring additional learnable parameters or data augmentation. Our results demonstrate that SC achieves the best performance in the aforementioned tasks.
  arXiv  Detail & Related papers  (2023-05-23T18:37:07Z)
• FisheyeHDK: Hyperbolic Deformable Kernel Learning for Ultra-Wide Field-of-View Image Recognition [0.3655021726150367]
  Conventional convolution neural networks (CNNs) trained on narrow Field-of-View (FoV) images are the state-of-the-art approaches for object recognition tasks. Some methods proposed the adaptation of CNNs to ultra-wide FoV images by learning deformable kernels. We demonstrate that learning the shape of convolution kernels in non-Euclidean spaces is better than existing deformable kernel methods.
  arXiv  Detail & Related papers  (2022-03-14T16:37:54Z)
• Adaptable Deformable Convolutions for Semantic Segmentation of Fisheye Images in Autonomous Driving Systems [4.231909978425546]
  We show that a CNN trained on standard images can be readily adapted to fisheye images. Our adaptation protocol mainly relies on modifying the support of the convolutions by using their deformable equivalents on top of pre-existing layers.
  arXiv  Detail & Related papers  (2021-02-19T22:47:44Z)
• PCLs: Geometry-aware Neural Reconstruction of 3D Pose with Perspective Crop Layers [111.55817466296402]
  We introduce Perspective Crop Layers (PCLs) - a form of perspective crop of the region of interest based on the camera geometry. PCLs deterministically remove the location-dependent perspective effects while leaving end-to-end training and the number of parameters of the underlying neural network. PCL offers an easy way to improve the accuracy of existing 3D reconstruction networks by making them geometry aware.
  arXiv  Detail & Related papers  (2020-11-27T08:48:43Z)
• Neural Ray Surfaces for Self-Supervised Learning of Depth and Ego-motion [51.19260542887099]
  We show that self-supervision can be used to learn accurate depth and ego-motion estimation without prior knowledge of the camera model. Inspired by the geometric model of Grossberg and Nayar, we introduce Neural Ray Surfaces (NRS), convolutional networks that represent pixel-wise projection rays. We demonstrate the use of NRS for self-supervised learning of visual odometry and depth estimation from raw videos obtained using a wide variety of camera systems.
  arXiv  Detail & Related papers  (2020-08-15T02:29:13Z)
• On Robustness and Transferability of Convolutional Neural Networks [147.71743081671508]
  Modern deep convolutional networks (CNNs) are often criticized for not generalizing under distributional shifts. We study the interplay between out-of-distribution and transfer performance of modern image classification CNNs for the first time. We find that increasing both the training set and model sizes significantly improve the distributional shift robustness.
  arXiv  Detail & Related papers  (2020-07-16T18:39:04Z)
• UnRectDepthNet: Self-Supervised Monocular Depth Estimation using a Generic Framework for Handling Common Camera Distortion Models [8.484676769284578]
  We propose a generic scale-aware self-supervised pipeline for estimating depth, euclidean distance, and visual odometry from unrectified monocular videos. The proposed algorithm is evaluated further on the KITTI rectified dataset, and we achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2020-07-13T20:35:05Z)
• PSConv: Squeezing Feature Pyramid into One Compact Poly-Scale Convolutional Layer [76.44375136492827]
  Convolutional Neural Networks (CNNs) are often scale-sensitive. We bridge this regret by exploiting multi-scale features in a finer granularity. The proposed convolution operation, named Poly-Scale Convolution (PSConv), mixes up a spectrum of dilation rates.
  arXiv  Detail & Related papers  (2020-07-13T05:14:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.