Arboretum: A Large Multimodal Dataset Enabling AI for Biodiversity

• URL: http://arxiv.org/abs/2406.17720v1
• Date: Tue, 25 Jun 2024 17:09:54 GMT
• Title: Arboretum: A Large Multimodal Dataset Enabling AI for Biodiversity
• Authors: Chih-Hsuan Yang, Benjamin Feuer, Zaki Jubery, Zi K. Deng, Andre Nakkab, Md Zahid Hasan, Shivani Chiranjeevi, Kelly Marshall, Nirmal Baishnab, Asheesh K Singh, Arti Singh, Soumik Sarkar, Nirav Merchant, Chinmay Hegde, Baskar Ganapathysubramanian,
• Abstract summary: This dataset includes 134.6 million images, surpassing existing datasets in scale by an order of magnitude. The dataset encompasses image-language paired data for a diverse set of species from birds (Aves), spiders/ticks/mites (Arachnida), insects (usha), plants (Plantae), fungus/mrooms (Fungi), snails (Mollusca), and snakes/Insectards (Reptilia)
• Score: 14.949271003068107
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Arboretum, the largest publicly accessible dataset designed to advance AI for biodiversity applications. This dataset, curated from the iNaturalist community science platform and vetted by domain experts to ensure accuracy, includes 134.6 million images, surpassing existing datasets in scale by an order of magnitude. The dataset encompasses image-language paired data for a diverse set of species from birds (Aves), spiders/ticks/mites (Arachnida), insects (Insecta), plants (Plantae), fungus/mushrooms (Fungi), snails (Mollusca), and snakes/lizards (Reptilia), making it a valuable resource for multimodal vision-language AI models for biodiversity assessment and agriculture research. Each image is annotated with scientific names, taxonomic details, and common names, enhancing the robustness of AI model training. We showcase the value of Arboretum by releasing a suite of CLIP models trained using a subset of 40 million captioned images. We introduce several new benchmarks for rigorous assessment, report accuracy for zero-shot learning, and evaluations across life stages, rare species, confounding species, and various levels of the taxonomic hierarchy. We anticipate that Arboretum will spur the development of AI models that can enable a variety of digital tools ranging from pest control strategies, crop monitoring, and worldwide biodiversity assessment and environmental conservation. These advancements are critical for ensuring food security, preserving ecosystems, and mitigating the impacts of climate change. Arboretum is publicly available, easily accessible, and ready for immediate use. Please see the \href{https://baskargroup.github.io/Arboretum/}{project website} for links to our data, models, and code.

Related papers

• Combining Observational Data and Language for Species Range Estimation [63.65684199946094]
  We propose a novel approach combining millions of citizen science species observations with textual descriptions from Wikipedia. Our framework maps locations, species, and text descriptions into a common space, enabling zero-shot range estimation from textual descriptions. Our approach also acts as a strong prior when combined with observational data, resulting in more accurate range estimation with less data.
  arXiv  Detail & Related papers  (2024-10-14T17:22:55Z)
• Insect Identification in the Wild: The AMI Dataset [35.41544843896443]
  Insects represent half of all global biodiversity, yet many of the world's insects are disappearing. Despite this crisis, data on insect diversity and abundance remain woefully inadequate. We provide the first large-scale machine learning benchmarks for fine-grained insect recognition.
  arXiv  Detail & Related papers  (2024-06-18T09:57:02Z)
• BioCLIP: A Vision Foundation Model for the Tree of Life [34.187429586642146]
  We release TreeOfLife-10M, the largest and most diverse ML-ready dataset of biology images. We then develop BioCLIP, a foundation model for the tree of life. We rigorously benchmark our approach on diverse fine-grained biology classification tasks.
  arXiv  Detail & Related papers  (2023-11-30T18:49:43Z)
• Multimodal Foundation Models for Zero-shot Animal Species Recognition in Camera Trap Images [57.96659470133514]
  Motion-activated camera traps constitute an efficient tool for tracking and monitoring wildlife populations across the globe. Supervised learning techniques have been successfully deployed to analyze such imagery, however training such techniques requires annotations from experts. Reducing the reliance on costly labelled data has immense potential in developing large-scale wildlife tracking solutions with markedly less human labor.
  arXiv  Detail & Related papers  (2023-11-02T08:32:00Z)
• SatBird: Bird Species Distribution Modeling with Remote Sensing and Citizen Science Data [68.2366021016172]
  We present SatBird, a satellite dataset of locations in the USA with labels derived from presence-absence observation data from the citizen science database eBird. We also provide a dataset in Kenya representing low-data regimes. We benchmark a set of baselines on our dataset, including SOTA models for remote sensing tasks.
  arXiv  Detail & Related papers  (2023-11-02T02:00:27Z)
• Species196: A One-Million Semi-supervised Dataset for Fine-grained Species Recognition [30.327642724046903]
  Species196 is a large-scale semi-supervised dataset of 196-category invasive species. It collects over 19K images with expert-level accurate annotations Species196-L, and 1.2M unlabeled images of invasive species Species196-U.
  arXiv  Detail & Related papers  (2023-09-25T14:46:01Z)
• A Step Towards Worldwide Biodiversity Assessment: The BIOSCAN-1M Insect Dataset [18.211840156134784]
  This paper presents a curated million-image dataset, primarily to train computer-vision models capable of providing image-based taxonomic assessment. The dataset also presents compelling characteristics, the study of which would be of interest to the broader machine learning community.
  arXiv  Detail & Related papers  (2023-07-19T20:54:08Z)
• Spatial Implicit Neural Representations for Global-Scale Species Mapping [72.92028508757281]
  Given a set of locations where a species has been observed, the goal is to build a model to predict whether the species is present or absent at any location. Traditional methods struggle to take advantage of emerging large-scale crowdsourced datasets. We use Spatial Implicit Neural Representations (SINRs) to jointly estimate the geographical range of 47k species simultaneously.
  arXiv  Detail & Related papers  (2023-06-05T03:36:01Z)
• Ensembles of Vision Transformers as a New Paradigm for Automated Classification in Ecology [0.0]
  We show that ensembles of Data-efficient image Transformers (DeiTs) significantly outperform the previous state of the art (SOTA) On all the data sets we test, we achieve a new SOTA, with a reduction of the error with respect to the previous SOTA ranging from 18.48% to 87.50%.
  arXiv  Detail & Related papers  (2022-03-03T14:16:22Z)
• Florida Wildlife Camera Trap Dataset [48.99466876948454]
  We introduce a challenging wildlife camera trap classification dataset collected from two different locations in Southwestern Florida. The dataset consists of 104,495 images featuring visually similar species, varying illumination conditions, skewed class distribution, and including samples of endangered species.
  arXiv  Detail & Related papers  (2021-06-23T18:53:15Z)
• Automatic image-based identification and biomass estimation of invertebrates [70.08255822611812]
  Time-consuming sorting and identification of taxa pose strong limitations on how many insect samples can be processed. We propose to replace the standard manual approach of human expert-based sorting and identification with an automatic image-based technology. We use state-of-the-art Resnet-50 and InceptionV3 CNNs for the classification task.
  arXiv  Detail & Related papers  (2020-02-05T21:38:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.