Mesoscopic photogrammetry with an unstabilized phone camera
- URL: http://arxiv.org/abs/2012.06044v1
- Date: Fri, 11 Dec 2020 00:09:18 GMT
- Title: Mesoscopic photogrammetry with an unstabilized phone camera
- Authors: Kevin C. Zhou, Colin Cooke, Jaehee Park, Ruobing Qian, Roarke
Horstmeyer, Joseph A. Izatt, Sina Farsiu
- Abstract summary: We present a feature-free photogrammetric computation technique that enables quantitative 3D mesoscopic (mm-scale height variation) imaging.
Our end-to-end, pixel-intensity-based approach jointly registers and stitches all the images by estimating a coaligned height map.
We also propose strategies for reducing time and memory, applicable to other multi-frame registration problems.
- Score: 8.210210271599134
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present a feature-free photogrammetric technique that enables quantitative
3D mesoscopic (mm-scale height variation) imaging with tens-of-micron accuracy
from sequences of images acquired by a smartphone at close range (several cm)
under freehand motion without additional hardware. Our end-to-end,
pixel-intensity-based approach jointly registers and stitches all the images by
estimating a coaligned height map, which acts as a pixel-wise radial
deformation field that orthorectifies each camera image to allow homographic
registration. The height maps themselves are reparameterized as the output of
an untrained encoder-decoder convolutional neural network (CNN) with the raw
camera images as the input, which effectively removes many reconstruction
artifacts. Our method also jointly estimates both the camera's dynamic 6D pose
and its distortion using a nonparametric model, the latter of which is
especially important in mesoscopic applications when using cameras not designed
for imaging at short working distances, such as smartphone cameras. We also
propose strategies for reducing computation time and memory, applicable to
other multi-frame registration problems. Finally, we demonstrate our method
using sequences of multi-megapixel images captured by an unstabilized
smartphone on a variety of samples (e.g., painting brushstrokes, circuit board,
seeds).
Related papers
- Single-image camera calibration with model-free distortion correction [0.0]
This paper proposes a method for estimating the complete set of calibration parameters from a single image of a planar speckle pattern covering the entire sensor.
The correspondence between image points and physical points on the calibration target is obtained using Digital Image Correlation.
At the end of the procedure, a dense and uniform model-free distortion map is obtained over the entire image.
arXiv Detail & Related papers (2024-03-02T16:51:35Z) - Learning Robust Multi-Scale Representation for Neural Radiance Fields
from Unposed Images [65.41966114373373]
We present an improved solution to the neural image-based rendering problem in computer vision.
The proposed approach could synthesize a realistic image of the scene from a novel viewpoint at test time.
arXiv Detail & Related papers (2023-11-08T08:18:23Z) - MC-NeRF: Multi-Camera Neural Radiance Fields for Multi-Camera Image Acquisition Systems [22.494866649536018]
Neural Radiance Fields (NeRF) use multi-view images for 3D scene representation, demonstrating remarkable performance.
Most previous NeRF-based methods assume a unique camera and rarely consider multi-camera scenarios.
We propose MC-NeRF, a method that enables joint optimization of both intrinsic and extrinsic parameters alongside NeRF.
arXiv Detail & Related papers (2023-09-14T16:40:44Z) - Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter
Correction [54.00007868515432]
Existing methods face challenges in estimating the accurate correction field due to the uniform velocity assumption.
We propose a geometry-based Quadratic Rolling Shutter (QRS) motion solver, which precisely estimates the high-order correction field of individual pixels.
Our method surpasses the state-of-the-art by +4.98, +0.77, and +4.33 of PSNR on Carla-RS, Fastec-RS, and BS-RSC datasets, respectively.
arXiv Detail & Related papers (2023-03-31T15:09:18Z) - Robust Camera Pose Refinement for Multi-Resolution Hash Encoding [19.655282240361835]
Multi-resolution hash encoding has been proposed to reduce the computational cost of neural renderings.
We propose a joint optimization algorithm to calibrate the camera pose and learn a geometric representation using efficient multi-resolution hash encoding.
arXiv Detail & Related papers (2023-02-03T06:49:27Z) - Robustifying the Multi-Scale Representation of Neural Radiance Fields [86.69338893753886]
We present a robust multi-scale neural radiance fields representation approach to overcome both real-world imaging issues.
Our method handles multi-scale imaging effects and camera-pose estimation problems with NeRF-inspired approaches.
We demonstrate, with examples, that for an accurate neural representation of an object from day-to-day acquired multi-view images, it is crucial to have precise camera-pose estimates.
arXiv Detail & Related papers (2022-10-09T11:46:45Z) - Towards Model Generalization for Monocular 3D Object Detection [57.25828870799331]
We present an effective unified camera-generalized paradigm (CGP) for Mono3D object detection.
We also propose the 2D-3D geometry-consistent object scaling strategy (GCOS) to bridge the gap via an instance-level augment.
Our method called DGMono3D achieves remarkable performance on all evaluated datasets and surpasses the SoTA unsupervised domain adaptation scheme.
arXiv Detail & Related papers (2022-05-23T23:05:07Z) - Self-Supervised Camera Self-Calibration from Video [34.35533943247917]
We propose a learning algorithm to regress per-sequence calibration parameters using an efficient family of general camera models.
Our procedure achieves self-calibration results with sub-pixel reprojection error, outperforming other learning-based methods.
arXiv Detail & Related papers (2021-12-06T19:42:05Z) - Self-Calibrating Neural Radiance Fields [68.64327335620708]
We jointly learn the geometry of the scene and the accurate camera parameters without any calibration objects.
Our camera model consists of a pinhole model, a fourth order radial distortion, and a generic noise model that can learn arbitrary non-linear camera distortions.
arXiv Detail & Related papers (2021-08-31T13:34:28Z) - Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled
Representation [57.11299763566534]
We present a solution to recover 3D pose from multi-view images captured with spatially calibrated cameras.
We exploit 3D geometry to fuse input images into a unified latent representation of pose, which is disentangled from camera view-points.
Our architecture then conditions the learned representation on camera projection operators to produce accurate per-view 2d detections.
arXiv Detail & Related papers (2020-04-05T12:52:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.