HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching

• URL: http://arxiv.org/abs/2007.12140v3
• Date: Thu, 8 Apr 2021 17:51:33 GMT
• Title: HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching
• Authors: Vladimir Tankovich, Christian H\"ane, Yinda Zhang, Adarsh Kowdle, Sean Fanello, Sofien Bouaziz
• Abstract summary: HITNet is a novel neural network architecture for real-time stereo matching. Our architecture is inherently multi-resolution allowing the propagation of information across different levels. At the time of writing, HITNet ranks 1st-3rd on all the metrics published on the ETH3D website for two view stereo.
• Score: 18.801346154045138
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents HITNet, a novel neural network architecture for real-time stereo matching. Contrary to many recent neural network approaches that operate on a full cost volume and rely on 3D convolutions, our approach does not explicitly build a volume and instead relies on a fast multi-resolution initialization step, differentiable 2D geometric propagation and warping mechanisms to infer disparity hypotheses. To achieve a high level of accuracy, our network not only geometrically reasons about disparities but also infers slanted plane hypotheses allowing to more accurately perform geometric warping and upsampling operations. Our architecture is inherently multi-resolution allowing the propagation of information across different levels. Multiple experiments prove the effectiveness of the proposed approach at a fraction of the computation required by state-of-the-art methods. At the time of writing, HITNet ranks 1st-3rd on all the metrics published on the ETH3D website for two view stereo, ranks 1st on most of the metrics among all the end-to-end learning approaches on Middlebury-v3, ranks 1st on the popular KITTI 2012 and 2015 benchmarks among the published methods faster than 100ms.

Related papers

• SVNet: Where SO(3) Equivariance Meets Binarization on Point Cloud Representation [65.4396959244269]
  The paper tackles the challenge by designing a general framework to construct 3D learning architectures. The proposed approach can be applied to general backbones like PointNet and DGCNN. Experiments on ModelNet40, ShapeNet, and the real-world dataset ScanObjectNN, demonstrated that the method achieves a great trade-off between efficiency, rotation, and accuracy.
  arXiv  Detail & Related papers  (2022-09-13T12:12:19Z)
• 3DVNet: Multi-View Depth Prediction and Volumetric Refinement [68.68537312256144]
  3DVNet is a novel multi-view stereo (MVS) depth-prediction method. Our key idea is the use of a 3D scene-modeling network that iteratively updates a set of coarse depth predictions. We show that our method exceeds state-of-the-art accuracy in both depth prediction and 3D reconstruction metrics.
  arXiv  Detail & Related papers  (2021-12-01T00:52:42Z)
• TSGCNet: Discriminative Geometric Feature Learning with Two-Stream GraphConvolutional Network for 3D Dental Model Segmentation [141.2690520327948]
  We propose a two-stream graph convolutional network (TSGCNet) to learn multi-view information from different geometric attributes. We evaluate our proposed TSGCNet on a real-patient dataset of dental models acquired by 3D intraoral scanners.
  arXiv  Detail & Related papers  (2020-12-26T08:02:56Z)
• Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks [87.50632573601283]
  We present a novel method for multi-view depth estimation from a single video. Our method achieves temporally coherent depth estimation results by using a novel Epipolar Spatio-Temporal (EST) transformer. To reduce the computational cost, inspired by recent Mixture-of-Experts models, we design a compact hybrid network.
  arXiv  Detail & Related papers  (2020-11-26T04:04:21Z)
• Densely connected multidilated convolutional networks for dense prediction tasks [25.75557472306157]
  We propose a novel CNN architecture called densely connected multidilated DenseNet (D3Net) D3Net involves a novel multidilated convolution that has different dilation factors in a single layer to model different resolutions simultaneously. Experiments on the image semantic segmentation task using Cityscapes and the audio source separation task using MUSDB18 show that the proposed method has superior performance over state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-11-21T05:15:12Z)
• SSP-Net: Scalable Sequential Pyramid Networks for Real-Time 3D Human Pose Regression [27.85790535227085]
  We propose a highly scalable convolutional neural network, end-to-end trainable, for real-time 3D human pose regression from still RGB images. Our network requires a single training procedure and is capable of producing its best predictions at 120 frames per second.
  arXiv  Detail & Related papers  (2020-09-04T03:43:24Z)
• Fully Dynamic Inference with Deep Neural Networks [19.833242253397206]
  Two compact networks, called Layer-Net (L-Net) and Channel-Net (C-Net), predict on a per-instance basis which layers or filters/channels are redundant and therefore should be skipped. On the CIFAR-10 dataset, LC-Net results in up to 11.9$times$ fewer floating-point operations (FLOPs) and up to 3.3% higher accuracy compared to other dynamic inference methods. On the ImageNet dataset, LC-Net achieves up to 1.4$times$ fewer FLOPs and up to 4.6% higher Top-1 accuracy than the other methods.
  arXiv  Detail & Related papers  (2020-07-29T23:17:48Z)
• Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking [54.58791377183574]
  Our novel hybrid recurrent multi-view stereo net consists of two core modules: 1) a light DRENet (Dense Reception Expanded) module to extract dense feature maps of original size with multi-scale context information, 2) a HU-LSTM (Hybrid U-LSTM) to regularize 3D matching volume into predicted depth map. Our method exhibits competitive performance to the state-of-the-art method while dramatically reduces memory consumption, which costs only $19.4%$ of R-MVSNet memory consumption.
  arXiv  Detail & Related papers  (2020-07-21T14:59:59Z)
• MSMD-Net: Deep Stereo Matching with Multi-scale and Multi-dimension Cost Volume [33.07553434167063]
  We propose MSMD-Net to construct multi-scale and multi-dimension cost volume. Our method shows strong domain-across generalization and outperforms best prior work by a margin with three or even five times faster speed.
  arXiv  Detail & Related papers  (2020-06-23T07:12:00Z)
• MetricUNet: Synergistic Image- and Voxel-Level Learning for Precise CT Prostate Segmentation via Online Sampling [66.01558025094333]
  We propose a two-stage framework, with the first stage to quickly localize the prostate region and the second stage to precisely segment the prostate. We introduce a novel online metric learning module through voxel-wise sampling in the multi-task network. Our method can effectively learn more representative voxel-level features compared with the conventional learning methods with cross-entropy or Dice loss.
  arXiv  Detail & Related papers  (2020-05-15T10:37:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.