Differentiable Uncalibrated Imaging

• URL: http://arxiv.org/abs/2211.10525v3
• Date: Wed, 20 Dec 2023 06:46:19 GMT
• Title: Differentiable Uncalibrated Imaging
• Authors: Sidharth Gupta, Konik Kothari, Valentin Debarnot, Ivan Dokmani\'c
• Abstract summary: We propose a differentiable imaging framework to address uncertainty in measurement coordinates such as sensor locations and projection angles. We apply implicit neural networks, also known as neural fields, which are naturally differentiable with respect to the input coordinates. Differentiability is key as it allows us to jointly fit a measurement representation, optimize over the uncertain measurement coordinates, and perform image reconstruction which in turn ensures consistent calibration.
• Score: 25.67247660827913
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a differentiable imaging framework to address uncertainty in measurement coordinates such as sensor locations and projection angles. We formulate the problem as measurement interpolation at unknown nodes supervised through the forward operator. To solve it we apply implicit neural networks, also known as neural fields, which are naturally differentiable with respect to the input coordinates. We also develop differentiable spline interpolators which perform as well as neural networks, require less time to optimize and have well-understood properties. Differentiability is key as it allows us to jointly fit a measurement representation, optimize over the uncertain measurement coordinates, and perform image reconstruction which in turn ensures consistent calibration. We apply our approach to 2D and 3D computed tomography, and show that it produces improved reconstructions compared to baselines that do not account for the lack of calibration. The flexibility of the proposed framework makes it easy to extend to almost arbitrary imaging problems.

Related papers

• 3D Equivariant Pose Regression via Direct Wigner-D Harmonics Prediction [50.07071392673984]
  Existing methods learn 3D rotations parametrized in the spatial domain using angles or quaternions. We propose a frequency-domain approach that directly predicts Wigner-D coefficients for 3D rotation regression. Our method achieves state-of-the-art results on benchmarks such as ModelNet10-SO(3) and PASCAL3D+.
  arXiv  Detail & Related papers  (2024-11-01T12:50:38Z)
• No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images [100.80376573969045]
  NoPoSplat is a feed-forward model capable of reconstructing 3D scenes parameterized by 3D Gaussians from multi-view images. Our model achieves real-time 3D Gaussian reconstruction during inference. This work makes significant advances in pose-free generalizable 3D reconstruction and demonstrates its applicability to real-world scenarios.
  arXiv  Detail & Related papers  (2024-10-31T17:58:22Z)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices. Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• Inertial Guided Uncertainty Estimation of Feature Correspondence in Visual-Inertial Odometry/SLAM [8.136426395547893]
  We propose a method to estimate the uncertainty of feature correspondence using an inertial guidance. We also demonstrate the feasibility of our approach by incorporating it into one of recent visual-inertial odometry/SLAM algorithms.
  arXiv  Detail & Related papers  (2023-11-07T04:56:29Z)
• GraphAlign: Enhancing Accurate Feature Alignment by Graph matching for Multi-Modal 3D Object Detection [7.743525134435137]
  LiDAR and cameras are complementary sensors for 3D object detection in autonomous driving. We present GraphAlign, a more accurate feature alignment strategy for 3D object detection by graph matching.
  arXiv  Detail & Related papers  (2023-10-12T12:06:31Z)
• Coordinate Quantized Neural Implicit Representations for Multi-view Reconstruction [28.910183274743872]
  We introduce neural implicit representations with quantized coordinates, which reduces the uncertainty and ambiguity in the field during optimization. We use discrete coordinates and their positional encodings to learn implicit functions through volume rendering. Our evaluations under the widely used benchmarks show our superiority over the state-of-the-art.
  arXiv  Detail & Related papers  (2023-08-21T20:27:33Z)
• Vanishing Point Estimation in Uncalibrated Images with Prior Gravity Direction [82.72686460985297]
  We tackle the problem of estimating a Manhattan frame. We derive two new 2-line solvers, one of which does not suffer from singularities affecting existing solvers. We also design a new non-minimal method, running on an arbitrary number of lines, to boost the performance in local optimization.
  arXiv  Detail & Related papers  (2023-08-21T13:03:25Z)
• NIKI: Neural Inverse Kinematics with Invertible Neural Networks for 3D Human Pose and Shape Estimation [53.25973084799954]
  We present NIKI (Neural Inverse Kinematics with Invertible Neural Network), which models bi-directional errors. NIKI can learn from both the forward and inverse processes with invertible networks.
  arXiv  Detail & Related papers  (2023-05-15T12:13:24Z)
• Error-Correcting Neural Networks for Two-Dimensional Curvature Computation in the Level-Set Method [0.0]
  We present an error-neural-modeling-based strategy for approximating two-dimensional curvature in the level-set method. Our main contribution is a redesigned hybrid solver that relies on numerical schemes to enable machine-learning operations on demand.
  arXiv  Detail & Related papers  (2022-01-22T05:14:40Z)
• PDC-Net+: Enhanced Probabilistic Dense Correspondence Network [161.76275845530964]
  Enhanced Probabilistic Dense Correspondence Network, PDC-Net+, capable of estimating accurate dense correspondences. We develop an architecture and an enhanced training strategy tailored for robust and generalizable uncertainty prediction. Our approach obtains state-of-the-art results on multiple challenging geometric matching and optical flow datasets.
  arXiv  Detail & Related papers  (2021-09-28T17:56:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.