Layout-guided Indoor Panorama Inpainting with Plane-aware Normalization

• URL: http://arxiv.org/abs/2301.05624v1
• Date: Fri, 13 Jan 2023 15:48:40 GMT
• Title: Layout-guided Indoor Panorama Inpainting with Plane-aware Normalization
• Authors: Chao-Chen Gao, Cheng-Hsiu Chen, Jheng-Wei Su, Hung-Kuo Chu
• Abstract summary: We present an end-to-end deep learning framework for indoor panoramic image inpainting. We exploit both the global and local context of indoor panorama during the inpainting process. Experimental results show that our work outperforms the current state-of-the-art methods on a public panoramic dataset.
• Score: 10.721512251097606
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present an end-to-end deep learning framework for indoor panoramic image inpainting. Although previous inpainting methods have shown impressive performance on natural perspective images, most fail to handle panoramic images, particularly indoor scenes, which usually contain complex structure and texture content. To achieve better inpainting quality, we propose to exploit both the global and local context of indoor panorama during the inpainting process. Specifically, we take the low-level layout edges estimated from the input panorama as a prior to guide the inpainting model for recovering the global indoor structure. A plane-aware normalization module is employed to embed plane-wise style features derived from the layout into the generator, encouraging local texture restoration from adjacent room structures (i.e., ceiling, floor, and walls). Experimental results show that our work outperforms the current state-of-the-art methods on a public panoramic dataset in both qualitative and quantitative evaluations. Our code is available at https://ericsujw.github.io/LGPN-net/

Related papers

• SPVLoc: Semantic Panoramic Viewport Matching for 6D Camera Localization in Unseen Environments [4.2603120588176635]
  SPVLoc is a global indoor localization method that accurately determines the 6D camera pose of a query image. Our approach employs a novel matching procedure to localize the perspective camera's viewport. It achieves superior localization accuracy compared to the state of the art methods and also estimates more degrees of freedom of the camera pose.
  arXiv  Detail & Related papers  (2024-04-16T12:55:15Z)
• Diagnostic Benchmark and Iterative Inpainting for Layout-Guided Image Generation [147.81509219686419]
  We propose a diagnostic benchmark for layout-guided image generation that examines four categories of spatial control skills: number, position, size, and shape. Next, we propose IterInpaint, a new baseline that generates foreground and background regions step-by-step via inpainting. We show comprehensive ablation studies on IterInpaint, including training task ratio, crop&paste vs. repaint, and generation order.
  arXiv  Detail & Related papers  (2023-04-13T16:58:33Z)
• Local-to-Global Panorama Inpainting for Locale-Aware Indoor Lighting Prediction [28.180205012351802]
  Predicting panoramic indoor lighting from a single perspective image is a fundamental but highly ill-posed problem in computer vision and graphics. Recent methods mostly rely on convolutional neural networks (CNNs) to fill the missing contents in the warped panorama. We propose a local-to-global strategy for large-scale panorama inpainting.
  arXiv  Detail & Related papers  (2023-03-18T06:18:49Z)
• Cylin-Painting: Seamless {360\textdegree} Panoramic Image Outpainting and Beyond [136.18504104345453]
  We present a Cylin-Painting framework that involves meaningful collaborations between inpainting and outpainting. The proposed algorithm can be effectively extended to other panoramic vision tasks, such as object detection, depth estimation, and image super-resolution.
  arXiv  Detail & Related papers  (2022-04-18T21:18:49Z)
• DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization [66.25948693095604]
  We propose a novel method for panoramic 3D scene understanding which recovers the 3D room layout and the shape, pose, position, and semantic category for each object from a single full-view panorama image. Experiments demonstrate that our method outperforms existing methods on panoramic scene understanding in terms of both geometry accuracy and object arrangement.
  arXiv  Detail & Related papers  (2021-08-24T13:55:29Z)
• Structure First Detail Next: Image Inpainting with Pyramid Generator [26.94101909283021]
  We propose to build a Pyramid Generator by stacking several sub-generators. Lower-layer sub-generators focus on restoring image structures while the higher-layer sub-generators emphasize image details. Our approach has a learning scheme of progressively increasing hole size, which allows it to restore large-hole images.
  arXiv  Detail & Related papers  (2021-06-16T16:00:16Z)
• Panoramic Panoptic Segmentation: Towards Complete Surrounding Understanding via Unsupervised Contrastive Learning [97.37544023666833]
  We introduce panoramic panoptic segmentation as the most holistic scene understanding. A complete surrounding understanding provides a maximum of information to the agent. We propose a framework which allows model training on standard pinhole images and transfers the learned features to a different domain.
  arXiv  Detail & Related papers  (2021-03-01T09:37:27Z)
• Perspective Plane Program Induction from a Single Image [85.28956922100305]
  We study the inverse graphics problem of inferring a holistic representation for natural images. We formulate this problem as jointly finding the camera pose and scene structure that best describe the input image. Our proposed framework, Perspective Plane Program Induction (P3I), combines search-based and gradient-based algorithms to efficiently solve the problem.
  arXiv  Detail & Related papers  (2020-06-25T21:18:58Z)
• Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed Scenes [54.836331922449666]
  We propose a Semantic Guidance and Evaluation Network (SGE-Net) to update the structural priors and the inpainted image. It utilizes semantic segmentation map as guidance in each scale of inpainting, under which location-dependent inferences are re-evaluated. Experiments on real-world images of mixed scenes demonstrated the superiority of our proposed method over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-03-15T17:49:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.