An End-to-End Network for Upright Adjustment of Panoramic Images

• URL: http://arxiv.org/abs/2304.05556v1
• Date: Wed, 12 Apr 2023 01:31:46 GMT
• Title: An End-to-End Network for Upright Adjustment of Panoramic Images
• Abstract summary: We propose an online end-to-end network for upright adjustment. Our network consists of three modules: orientation estimation, LUT online generation, and upright reconstruction. In terms of image reconstruction, we have achieved the first real-time online upright reconstruction of panoramic images using deep learning networks.
• Score: 4.8866383302403955
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nowadays, panoramic images can be easily obtained by panoramic cameras. However, when the panoramic camera orientation is tilted, a non-upright panoramic image will be captured. Existing upright adjustment models focus on how to estimate more accurate camera orientation, and attribute image reconstruction to offline or post-processing tasks. To this end, we propose an online end-to-end network for upright adjustment. Our network is designed to reconstruct the image while finding the angle. Our network consists of three modules: orientation estimation, LUT online generation, and upright reconstruction. Direction estimation estimates the tilt angle of the panoramic image. Then, a converter block with upsampling function is designed to generate angle to LUT. This module can output corresponding online LUT for different input angles. Finally, a lightweight generative adversarial network (GAN) aims to generate upright images from shallow features. The experimental results show that in terms of angles, we have improved the accuracy of small angle errors. In terms of image reconstruction, In image reconstruction, we have achieved the first real-time online upright reconstruction of panoramic images using deep learning networks.

Related papers

• ReX-Shot: Single-Image Rephotography via Geometry- and Camera-Grounded Generation [77.23728599741166]
  Single-image rephotography aims to synthesize new shots of a scene from a single reference image with specified viewpoints, focal lengths, and photographic effects.<n>Existing methods treat these factors separately and struggle under joint control.<n>ReX-Shot is the first unified framework to jointly control viewpoint, focal length, and parameterized photographic effects from a single image.
  arXiv  Detail & Related papers  (2026-08-19T06:34:20Z)
• Unified Panoramic Geometry Estimation via Multi-View Foundation Models [22.160964470897003]
  PaGeR (Panoramic Geometry Reconstruction) is a framework to lift powerful 3D foundation models to the panorama domain.<n>Our strategy is to start from a pre-trained transformer for 3D reconstruction and turn it into a unified high-performance model.<n>We extensively test our method in both indoor and outdoor environments and find that it delivers state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-05-25T22:32:42Z)
• World-Shaper: A Unified Framework for 360° Panoramic Editing [57.174341220144605]
  Existing perspective-based image editing methods fail to model the spatial structure of panoramas.<n>We present World-Shaper, a unified geometry-aware framework that bridges panoramic generation and editing within a single editing-centric design.<n>Our method achieves superior geometric consistency, editing fidelity, and text controllability compared to SOTA methods.
  arXiv  Detail & Related papers  (2026-01-30T19:38:54Z)
• A Deep Single Image Rectification Approach for Pan-Tilt-Zoom Cameras [21.404790627439954]
  This paper presents a Forward Distortion and Backward Warping Network (FDBWNet), a novel framework for wide-grained image rectification. It begins by using a forward distortion model synthesize barreldistorted images, reducing pixel redundancy and preventing blur. The network employs a pyramid context with attention mechanisms generate backward warping flows containing geometric details.
  arXiv  Detail & Related papers  (2025-04-09T15:19:38Z)
• Masked Angle-Aware Autoencoder for Remote Sensing Images [43.65954772044253]
  This paper proposes the Masked Angle-Aware Autoencoder (MA3E) to perceive and learn angles during pre-training. We show that MA3E demonstrates more competitive performance than existing pre-training methods on seven different RS image datasets in three downstream tasks.
  arXiv  Detail & Related papers  (2024-08-04T07:12:59Z)
• Stretched sinograms for limited-angle tomographic reconstruction with neural networks [1.52292571922932]
  We present a direct method for limited angle tomographic reconstruction using convolutional networks. The key to our method is to first stretch every tilt view in the direction to the tilt axis by the secant of the tilt angle. We train our networks by minimizing the mean squared error between the network's generated reconstruction and a ground truth 3-D volume.
  arXiv  Detail & Related papers  (2023-06-16T22:35:12Z)
• RelPose++: Recovering 6D Poses from Sparse-view Observations [66.6922660401558]
  We address the task of estimating 6D camera poses from sparse-view image sets (2-8 images) We build on the recent RelPose framework which learns a network that infers distributions over relative rotations over image pairs. Our final system results in large improvements in 6D pose prediction over prior art on both seen and unseen object categories.
  arXiv  Detail & Related papers  (2023-05-08T17:59:58Z)
• RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning [62.86400614141706]
  We propose a new learning model, i.e., Rectangling Rectification Network (RecRecNet) Our model can flexibly warp the source structure to the target domain and achieves an end-to-end unsupervised deformation. Experiments show the superiority of our solution over the compared methods on both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2023-01-04T15:12:57Z)
• Deep Rotation Correction without Angle Prior [57.76737888499145]
  We propose a new and practical task, named Rotation Correction, to automatically correct the tilt with high content fidelity. This task can be easily integrated into image editing applications, allowing users to correct the rotated images without any manual operations. We leverage a neural network to predict the optical flows that can warp the tilted images to be perceptually horizontal.
  arXiv  Detail & Related papers  (2022-07-07T02:46:27Z)
• PCLs: Geometry-aware Neural Reconstruction of 3D Pose with Perspective Crop Layers [111.55817466296402]
  We introduce Perspective Crop Layers (PCLs) - a form of perspective crop of the region of interest based on the camera geometry. PCLs deterministically remove the location-dependent perspective effects while leaving end-to-end training and the number of parameters of the underlying neural network. PCL offers an easy way to improve the accuracy of existing 3D reconstruction networks by making them geometry aware.
  arXiv  Detail & Related papers  (2020-11-27T08:48:43Z)
• Quantization in Relative Gradient Angle Domain For Building Polygon Estimation [88.80146152060888]
  CNN approaches often generate imprecise building morphologies including noisy edges and round corners. We propose a module that uses prior knowledge of building corners to create angular and concise building polygons from CNN segmentation outputs. Experimental results demonstrate that our method refines CNN output from a rounded approximation to a more clear-cut angular shape of the building footprint.
  arXiv  Detail & Related papers  (2020-07-10T21:33:06Z)
• Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images [59.906948203578544]
  We introduce a novel learning-based method to reconstruct the high-quality geometry and complex, spatially-varying BRDF of an arbitrary object. We first estimate per-view depth maps using a deep multi-view stereo network. These depth maps are used to coarsely align the different views. We propose a novel multi-view reflectance estimation network architecture.
  arXiv  Detail & Related papers  (2020-03-27T21:28:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.