Fast Omni-Directional Image Super-Resolution: Adapting the Implicit Image Function with Pixel and Semantic-Wise Spherical Geometric Priors

• URL: http://arxiv.org/abs/2502.05902v1
• Date: Sun, 09 Feb 2025 13:37:50 GMT
• Title: Fast Omni-Directional Image Super-Resolution: Adapting the Implicit Image Function with Pixel and Semantic-Wise Spherical Geometric Priors
• Authors: Xuelin Shen, Yitong Wang, Silin Zheng, Kang Xiao, Wenhan Yang, Xu Wang,
• Abstract summary: This paper proposes a new ODI-SR model characterized by its capacity to perform Fast and Arbitrary-scale ODI-SR processes. The proposed FAOR outperforms the state-of-the-art ODI-SR models with a much faster inference speed.
• Score: 38.580815475638595
• License:
• Abstract: In the context of Omni-Directional Image (ODI) Super-Resolution (SR), the unique challenge arises from the non-uniform oversampling characteristics caused by EquiRectangular Projection (ERP). Considerable efforts in designing complex spherical convolutions or polyhedron reprojection offer significant performance improvements but at the expense of cumbersome processing procedures and slower inference speeds. Under these circumstances, this paper proposes a new ODI-SR model characterized by its capacity to perform Fast and Arbitrary-scale ODI-SR processes, denoted as FAOR. The key innovation lies in adapting the implicit image function from the planar image domain to the ERP image domain by incorporating spherical geometric priors at both the latent representation and image reconstruction stages, in a low-overhead manner. Specifically, at the latent representation stage, we adopt a pair of pixel-wise and semantic-wise sphere-to-planar distortion maps to perform affine transformations on the latent representation, thereby incorporating it with spherical properties. Moreover, during the image reconstruction stage, we introduce a geodesic-based resampling strategy, aligning the implicit image function with spherical geometrics without introducing additional parameters. As a result, the proposed FAOR outperforms the state-of-the-art ODI-SR models with a much faster inference speed. Extensive experimental results and ablation studies have demonstrated the effectiveness of our design.

Related papers

• Effective Diffusion Transformer Architecture for Image Super-Resolution [63.254644431016345]
  We design an effective diffusion transformer for image super-resolution (DiT-SR) In practice, DiT-SR leverages an overall U-shaped architecture, and adopts a uniform isotropic design for all the transformer blocks. We analyze the limitation of the widely used AdaLN, and present a frequency-adaptive time-step conditioning module.
  arXiv  Detail & Related papers  (2024-09-29T07:14:16Z)
• Invertible Neural Warp for NeRF [29.00183106905031]
  This paper tackles the simultaneous optimization of pose and Neural Radiance Fields (NeRF) We propose a novel over parameterized representation that models camera poses as learnable rigid warp functions. We present results on synthetic and real-world datasets, and demonstrate that our approach outperforms existing baselines in terms of pose estimation and high-fidelity reconstruction.
  arXiv  Detail & Related papers  (2024-07-17T07:14:08Z)
• Geometric Distortion Guided Transformer for Omnidirectional Image Super-Resolution [26.819508321968108]
  We introduce a novel Geometric Distortion Guided Transformer for Omnidirectional image Super-Resolution (GDGT-OSR) Specifically, a distortion modulated rectangle-window self-attention mechanism, integrated with deformable self-attention, is proposed to better perceive the distortion. We show that the new GDGT-OSR outperforms methods in existing literature.
  arXiv  Detail & Related papers  (2024-06-16T09:38:33Z)
• Efficient Visual State Space Model for Image Deblurring [83.57239834238035]
  Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration. We propose a simple yet effective visual state space model (EVSSM) for image deblurring.
  arXiv  Detail & Related papers  (2024-05-23T09:13:36Z)
• Physics-Inspired Degradation Models for Hyperspectral Image Fusion [61.743696362028246]
  Most fusion methods solely focus on the fusion algorithm itself and overlook the degradation models. We propose physics-inspired degradation models (PIDM) to model the degradation of LR-HSI and HR-MSI. Our proposed PIDM can boost the fusion performance of existing fusion methods in practical scenarios.
  arXiv  Detail & Related papers  (2024-02-04T09:07:28Z)
• BID-NeRF: RGB-D image pose estimation with inverted Neural Radiance Fields [0.0]
  We aim to improve the Inverted Neural Radiance Fields (iNeRF) algorithm which defines the image pose estimation problem as a NeRF based iterative linear optimization. NeRFs are novel neural space representation models that can synthesize photorealistic novel views of real-world scenes or objects.
  arXiv  Detail & Related papers  (2023-10-05T14:27:06Z)
• High-Dimensional MR Reconstruction Integrating Subspace and Adaptive Generative Models [21.719520686704474]
  We present a novel method that integrates subspace modeling with an adaptive generative image prior for high-dimensional MR image reconstruction. We evaluated the utility of the proposed method in two high-dimensional imaging applications: accelerated MR parameter mapping and high-resolution MRSI.
  arXiv  Detail & Related papers  (2023-06-14T16:43:14Z)
• Hierarchical Similarity Learning for Aliasing Suppression Image Super-Resolution [64.15915577164894]
  A hierarchical image super-resolution network (HSRNet) is proposed to suppress the influence of aliasing. HSRNet achieves better quantitative and visual performance than other works, and remits the aliasing more effectively.
  arXiv  Detail & Related papers  (2022-06-07T14:55:32Z)
• Implicit Neural Representation Learning for Hyperspectral Image Super-Resolution [0.0]
  Implicit Neural Representations (INRs) are making strides as a novel and effective representation. We propose a novel HSI reconstruction model based on INR which represents HSI by a continuous function mapping a spatial coordinate to its corresponding spectral radiance values.
  arXiv  Detail & Related papers  (2021-12-20T14:07:54Z)
• Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications [78.63280020581662]
  A novel convolutional neural network (CNN)-based framework is developed for light field reconstruction from a sparse set of views. We demonstrate the high performance and robustness of the proposed framework compared with state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2021-03-24T08:16:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.