Efficient Neighbourhood Consensus Networks via Submanifold Sparse Convolutions

• URL: http://arxiv.org/abs/2004.10566v1
• Date: Wed, 22 Apr 2020 13:37:36 GMT
• Title: Efficient Neighbourhood Consensus Networks via Submanifold Sparse Convolutions
• Authors: Ignacio Rocco, Relja Arandjelovi\'c, Josef Sivic
• Abstract summary: We adopt the recent Neighbourhood Consensus Networks that have demonstrated promising performance for difficult correspondence problems. We propose modifications to overcome their main limitations: large memory consumption, large inference time and poorly localised correspondences. Our proposed modifications can reduce the memory footprint and execution time more than $10times$, with equivalent results.
• Score: 41.43309123350792
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work we target the problem of estimating accurately localised correspondences between a pair of images. We adopt the recent Neighbourhood Consensus Networks that have demonstrated promising performance for difficult correspondence problems and propose modifications to overcome their main limitations: large memory consumption, large inference time and poorly localised correspondences. Our proposed modifications can reduce the memory footprint and execution time more than $10\times$, with equivalent results. This is achieved by sparsifying the correlation tensor containing tentative matches, and its subsequent processing with a 4D CNN using submanifold sparse convolutions. Localisation accuracy is significantly improved by processing the input images in higher resolution, which is possible due to the reduced memory footprint, and by a novel two-stage correspondence relocalisation module. The proposed Sparse-NCNet method obtains state-of-the-art results on the HPatches Sequences and InLoc visual localisation benchmarks, and competitive results in the Aachen Day-Night benchmark.

Related papers

• AdaBM: On-the-Fly Adaptive Bit Mapping for Image Super-Resolution [53.23803932357899]
  We introduce the first on-the-fly adaptive quantization framework that accelerates the processing time from hours to seconds. We achieve competitive performance with the previous adaptive quantization methods, while the processing time is accelerated by x2000.
  arXiv  Detail & Related papers  (2024-04-04T08:37:27Z)
• Scale-Aware Crowd Count Network with Annotation Error Correction [19.036693336902662]
  Traditional crowd counting networks suffer from information loss when feature maps are downsized through pooling layers. We propose a Scale-Aware Crowd Counting Network (SACC-Net) that introduces a scale-aware'' architecture with error-correcting capabilities of noisy annotations.
  arXiv  Detail & Related papers  (2023-12-28T01:36:38Z)
• Deep Multi-Threshold Spiking-UNet for Image Processing [51.88730892920031]
  This paper introduces the novel concept of Spiking-UNet for image processing, which combines the power of Spiking Neural Networks (SNNs) with the U-Net architecture. To achieve an efficient Spiking-UNet, we face two primary challenges: ensuring high-fidelity information propagation through the network via spikes and formulating an effective training strategy. Experimental results show that, on image segmentation and denoising, our Spiking-UNet achieves comparable performance to its non-spiking counterpart.
  arXiv  Detail & Related papers  (2023-07-20T16:00:19Z)
• SqueezeLLM: Dense-and-Sparse Quantization [80.32162537942138]
  Main bottleneck for generative inference with LLMs is memory bandwidth, rather than compute, for single batch inference. We introduce SqueezeLLM, a post-training quantization framework that enables lossless compression to ultra-low precisions of up to 3-bit. Our framework incorporates two novel ideas: (i) sensitivity-based non-uniform quantization, which searches for the optimal bit precision assignment based on second-order information; and (ii) the Dense-and-Sparse decomposition that stores outliers and sensitive weight values in an efficient sparse format.
  arXiv  Detail & Related papers  (2023-06-13T08:57:54Z)
• Improving Transformer-based Image Matching by Cascaded Capturing Spatially Informative Keypoints [44.90917854990362]
  We propose a transformer-based cascade matching model -- Cascade feature Matching TRansformer (CasMTR) We use a simple yet effective Non-Maximum Suppression (NMS) post-process to filter keypoints through the confidence map. CasMTR achieves state-of-the-art performance in indoor and outdoor pose estimation as well as visual localization.
  arXiv  Detail & Related papers  (2023-03-06T04:32:34Z)
• DeepSplit: Scalable Verification of Deep Neural Networks via Operator Splitting [70.62923754433461]
  Analyzing the worst-case performance of deep neural networks against input perturbations amounts to solving a large-scale non- optimization problem. We propose a novel method that can directly solve a convex relaxation of the problem to high accuracy, by splitting it into smaller subproblems that often have analytical solutions.
  arXiv  Detail & Related papers  (2021-06-16T20:43:49Z)
• $\mathbb{X}$Resolution Correspondence Networks [15.214155342197474]
  In this paper, we aim at establishing accurate dense correspondences between a pair of images with overlapping field of view under challenging illumination variation, viewpoint changes, and style differences.
  arXiv  Detail & Related papers  (2020-12-17T18:57:58Z)
• Improving accuracy and speeding up Document Image Classification through parallel systems [4.102028235659611]
  We show in the RVL-CDIP dataset that we can improve previous results with a much lighter model. We present an ensemble pipeline which is able to boost solely image input. Lastly, we expose the training performance differences between PyTorch and Deep Learning frameworks.
  arXiv  Detail & Related papers  (2020-06-16T13:36:07Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)
• Compression of descriptor models for mobile applications [26.498907514590165]
  We evaluate the computational cost, model size, and matching accuracy tradeoffs for deep neural networks. We observe a significant redundancy in the learned weights, which we exploit through the use of depthwise separable layers. We propose the Convolution-Depthwise-Pointwise(CDP) layer, which provides a means of interpolating between the standard and depthwise separable convolutions.
  arXiv  Detail & Related papers  (2020-01-09T17:00:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.