FloorLevel-Net: Recognizing Floor-Level Lines with Height-Attention-Guided Multi-task Learning

• URL: http://arxiv.org/abs/2107.02462v1
• Date: Tue, 6 Jul 2021 08:17:59 GMT
• Title: FloorLevel-Net: Recognizing Floor-Level Lines with Height-Attention-Guided Multi-task Learning
• Authors: Mengyang Wu, Wei Zeng, Chi-Wing Fu
• Abstract summary: This work tackles the problem of locating floor-level lines in street-view images, using a supervised deep learning approach. We first compile a new dataset and develop a new data augmentation scheme to synthesize training samples. Next, we design FloorLevel-Net, a multi-task learning network that associates explicit features of building facades and implicit floor-level lines.
• Score: 49.30194762653723
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The ability to recognize the position and order of the floor-level lines that divide adjacent building floors can benefit many applications, for example, urban augmented reality (AR). This work tackles the problem of locating floor-level lines in street-view images, using a supervised deep learning approach. Unfortunately, very little data is available for training such a network $-$ current street-view datasets contain either semantic annotations that lack geometric attributes, or rectified facades without perspective priors. To address this issue, we first compile a new dataset and develop a new data augmentation scheme to synthesize training samples by harassing (i) the rich semantics of existing rectified facades and (ii) perspective priors of buildings in diverse street views. Next, we design FloorLevel-Net, a multi-task learning network that associates explicit features of building facades and implicit floor-level lines, along with a height-attention mechanism to help enforce a vertical ordering of floor-level lines. The generated segmentations are then passed to a second-stage geometry post-processing to exploit self-constrained geometric priors for plausible and consistent reconstruction of floor-level lines. Quantitative and qualitative evaluations conducted on assorted facades in existing datasets and street views from Google demonstrate the effectiveness of our approach. Also, we present context-aware image overlay results and show the potentials of our approach in enriching AR-related applications.

Related papers

• Fine-Grained Building Function Recognition from Street-View Images via Geometry-Aware Semi-Supervised Learning [18.432786227782803]
  We propose a geometry-aware semi-supervised framework for fine-grained building function recognition. We use geometric relationships among multi-source data to enhance pseudo-label accuracy in semi-supervised learning. Our proposed framework exhibits superior performance in fine-grained functional recognition of buildings.
  arXiv  Detail & Related papers  (2024-08-18T12:48:48Z)
• RoIPoly: Vectorized Building Outline Extraction Using Vertex and Logit Embeddings [5.093758132026397]
  We propose a novel query-based approach for extracting building outlines from aerial or satellite imagery. We formulate each polygon as a query and constrain the query attention on the most relevant regions of a potential building. We evaluate our method on the vectorized building outline extraction dataset CrowdAI and the 2D floorplan reconstruction dataset Structured3D.
  arXiv  Detail & Related papers  (2024-07-20T16:12:51Z)
• Open-Vocabulary Camouflaged Object Segmentation [66.94945066779988]
  We introduce a new task, open-vocabulary camouflaged object segmentation (OVCOS) We construct a large-scale complex scene dataset (textbfOVCamo) containing 11,483 hand-selected images with fine annotations and corresponding object classes. By integrating the guidance of class semantic knowledge and the supplement of visual structure cues from the edge and depth information, the proposed method can efficiently capture camouflaged objects.
  arXiv  Detail & Related papers  (2023-11-19T06:00:39Z)
• Segment Any Building [8.12405696290333]
  This manuscript accentuates the potency of harnessing diversified datasets in tandem with cutting-edge representation learning paradigms for building segmentation in such images. Our avant-garde joint training regimen underscores the merit of our approach, bearing significant implications in pivotal domains such as urban infrastructural development, disaster mitigation strategies, and ecological surveillance. The outcomes of this research both fortify the foundations for ensuing scholarly pursuits and presage a horizon replete with innovative applications in the discipline of building segmentation.
  arXiv  Detail & Related papers  (2023-10-02T12:49:20Z)
• Semi-supervised Learning from Street-View Images and OpenStreetMap for Automatic Building Height Estimation [59.6553058160943]
  We propose a semi-supervised learning (SSL) method of automatically estimating building height from Mapillary SVI and OpenStreetMap data. The proposed method leads to a clear performance boosting in estimating building heights with a Mean Absolute Error (MAE) around 2.1 meters. The preliminary result is promising and motivates our future work in scaling up the proposed method based on low-cost VGI data.
  arXiv  Detail & Related papers  (2023-07-05T18:16:30Z)
• 3D Line Mapping Revisited [86.13455066577657]
  LIMAP is a library for 3D line mapping that robustly and efficiently creates 3D line maps from multi-view imagery. Our code integrates seamlessly with existing point-based Structure-from-Motion methods. Our robust 3D line maps also open up new research directions.
  arXiv  Detail & Related papers  (2023-03-30T16:14:48Z)
• SOLD2: Self-supervised Occlusion-aware Line Description and Detection [95.8719432775724]
  We introduce the first joint detection and description of line segments in a single deep network. Our method does not require any annotated line labels and can therefore generalize to any dataset. We evaluate our approach against previous line detection and description methods on several multi-view datasets.
  arXiv  Detail & Related papers  (2021-04-07T19:27:17Z)
• Campus3D: A Photogrammetry Point Cloud Benchmark for Hierarchical Understanding of Outdoor Scene [76.4183572058063]
  We present a richly-annotated 3D point cloud dataset for multiple outdoor scene understanding tasks. The dataset has been point-wisely annotated with both hierarchical and instance-based labels. We formulate a hierarchical learning problem for 3D point cloud segmentation and propose a measurement evaluating consistency across various hierarchies.
  arXiv  Detail & Related papers  (2020-08-11T19:10:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.