FoMo-Bench: a multi-modal, multi-scale and multi-task Forest Monitoring Benchmark for remote sensing foundation models

• URL: http://arxiv.org/abs/2312.10114v2
• Date: Wed, 27 Mar 2024 09:00:54 GMT
• Title: FoMo-Bench: a multi-modal, multi-scale and multi-task Forest Monitoring Benchmark for remote sensing foundation models
• Authors: Nikolaos Ioannis Bountos, Arthur Ouaknine, David Rolnick,
• Abstract summary: We present the first unified Forest Monitoring Benchmark (FoMo-Bench) FoMo-Bench consists of 15 diverse datasets encompassing satellite, aerial, and inventory data. To further enhance the diversity of tasks and geographies represented in FoMo-Bench, we introduce a novel global dataset, TalloS.
• Score: 24.141443217910986
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Forests are an essential part of Earth's ecosystems and natural systems, as well as providing services on which humanity depends, yet they are rapidly changing as a result of land use decisions and climate change. Understanding and mitigating negative effects requires parsing data on forests at global scale from a broad array of sensory modalities, and recently many such problems have been approached using machine learning algorithms for remote sensing. To date, forest-monitoring problems have largely been addressed in isolation. Inspired by the rise of foundation models for computer vision and remote sensing, we here present the first unified Forest Monitoring Benchmark (FoMo-Bench). FoMo-Bench consists of 15 diverse datasets encompassing satellite, aerial, and inventory data, covering a variety of geographical regions, and including multispectral, red-green-blue, synthetic aperture radar (SAR) and LiDAR data with various temporal, spatial and spectral resolutions. FoMo-Bench includes multiple types of forest-monitoring tasks, spanning classification, segmentation, and object detection. To further enhance the diversity of tasks and geographies represented in FoMo-Bench, we introduce a novel global dataset, TalloS, combining satellite imagery with ground-based annotations for tree species classification, encompassing 1,000+ categories across multiple hierarchical taxonomic levels (species, genus, family). Finally, we propose FoMo-Net, a baseline foundation model with the capacity to process any combination of commonly used spectral bands in remote sensing, across diverse ground sampling distances and geographical locations worldwide. This work aims to inspire research collaborations between machine learning and forest biology researchers in exploring scalable multi-modal and multi-task models for forest monitoring. All code and data will be made publicly available.

Related papers

• Towards Vision-Language Geo-Foundation Model: A Survey [65.70547895998541]
  Vision-Language Foundation Models (VLFMs) have made remarkable progress on various multimodal tasks. This paper thoroughly reviews VLGFMs, summarizing and analyzing recent developments in the field.
  arXiv  Detail & Related papers  (2024-06-13T17:57:30Z)
• Planted: a dataset for planted forest identification from multi-satellite time series [23.822292894884427]
  We present a dataset consisting of data from five public satellites for recognizing forest plantations and planted tree species across the globe. The dataset, named PlantD, includes over 2M examples of 64 tree label classes (46 genera and 40 species), distributed among 41 countries.
  arXiv  Detail & Related papers  (2024-05-24T15:49:00Z)
• MTMMC: A Large-Scale Real-World Multi-Modal Camera Tracking Benchmark [63.878793340338035]
  Multi-target multi-camera tracking is a crucial task that involves identifying and tracking individuals over time using video streams from multiple cameras. Existing datasets for this task are either synthetically generated or artificially constructed within a controlled camera network setting. We present MTMMC, a real-world, large-scale dataset that includes long video sequences captured by 16 multi-modal cameras in two different environments.
  arXiv  Detail & Related papers  (2024-03-29T15:08:37Z)
• Forest Inspection Dataset for Aerial Semantic Segmentation and Depth Estimation [6.635604919499181]
  We introduce a new large aerial dataset for forest inspection. It contains both real-world and virtual recordings of natural environments. We develop a framework to assess the deforestation degree of an area.
  arXiv  Detail & Related papers  (2024-03-11T11:26:44Z)
• LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environment [59.320414108383055]
  We present LiveHPS, a novel single-LiDAR-based approach for scene-level human pose and shape estimation. We propose a huge human motion dataset, named FreeMotion, which is collected in various scenarios with diverse human poses.
  arXiv  Detail & Related papers  (2024-02-27T03:08:44Z)
• SatBird: Bird Species Distribution Modeling with Remote Sensing and Citizen Science Data [68.2366021016172]
  We present SatBird, a satellite dataset of locations in the USA with labels derived from presence-absence observation data from the citizen science database eBird. We also provide a dataset in Kenya representing low-data regimes. We benchmark a set of baselines on our dataset, including SOTA models for remote sensing tasks.
  arXiv  Detail & Related papers  (2023-11-02T02:00:27Z)
• OpenForest: A data catalogue for machine learning in forest monitoring [22.94664416295411]
  Advancing forest monitoring offers advantages in mitigating human impacts and enhancing our comprehension of forest composition. We provide a comprehensive overview of 86 open access forest datasets across spatial scales. These datasets are grouped in OpenForest, a dynamic catalogue open to contributions that strives to reference all available open access forest datasets.
  arXiv  Detail & Related papers  (2023-11-01T03:59:20Z)
• Rapid Deforestation and Burned Area Detection using Deep Multimodal Learning on Satellite Imagery [3.8073142980733]
  Deforestation estimation and fire detection in the Amazon forest poses a significant challenge due to the vast size of the area. multimodal satellite imagery and remote sensing offer a promising solution for estimating deforestation and detecting wildfire in the Amazonia region. This research paper introduces a new curated dataset and a deep learning-based approach to solve these problems using convolutional neural networks (CNNs) and comprehensive data processing techniques.
  arXiv  Detail & Related papers  (2023-07-10T21:49:30Z)
• Bird Distribution Modelling using Remote Sensing and Citizen Science data [31.375576105932442]
  Climate change is a major driver of biodiversity loss. There are significant knowledge gaps about the distribution of species. We propose an approach leveraging computer vision to improve species distribution modelling.
  arXiv  Detail & Related papers  (2023-05-01T20:27:11Z)
• A General Purpose Neural Architecture for Geospatial Systems [142.43454584836812]
  We present a roadmap towards the construction of a general-purpose neural architecture (GPNA) with a geospatial inductive bias. We envision how such a model may facilitate cooperation between members of the community.
  arXiv  Detail & Related papers  (2022-11-04T09:58:57Z)
• Multiple-environment Self-adaptive Network for Aerial-view Geo-localization [85.52750931345287]
  Aerial-view geo-localization tends to determine an unknown position through matching the drone-view image with the geo-tagged satellite-view image. We propose a Multiple-environment Self-adaptive Network (MuSe-Net) to adjust the domain shift caused by environmental changing. In particular, MuSe-Net employs a two-branch neural network containing one multiple-environment style extraction network and one self-adaptive feature extraction network.
  arXiv  Detail & Related papers  (2022-04-18T16:04:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.