FathomVerse: A community science dataset for ocean animal discovery

• URL: http://arxiv.org/abs/2412.01701v1
• Date: Mon, 02 Dec 2024 16:49:20 GMT
• Title: FathomVerse: A community science dataset for ocean animal discovery
• Authors: Genevieve Patterson, Joost Daniels, Benjamin Woodward, Kevin Barnard, Giovanna Sainz, Lonny Lundsten, Kakani Katija,
• Abstract summary: We present the FathomVerse v0 dataset to push the limits of our field by exploring animals that rarely come in contact with people in the deep sea. The dataset consists of 3843 images with 8092 bounding boxes from 12 distinct morphological groups recorded at two locations on the deep seafloor. It features visually perplexing scenarios such as an octopus intertwined with a sea star, and confounding categories like vampire squids and sea spiders.
• Score: 0.7336679261573507
• License:
• Abstract: Can computer vision help us explore the ocean? The ultimate challenge for computer vision is to recognize any visual phenomena, more than only the objects and animals humans encounter in their terrestrial lives. Previous datasets have explored everyday objects and fine-grained categories humans see frequently. We present the FathomVerse v0 detection dataset to push the limits of our field by exploring animals that rarely come in contact with people in the deep sea. These animals present a novel vision challenge. The FathomVerse v0 dataset consists of 3843 images with 8092 bounding boxes from 12 distinct morphological groups recorded at two locations on the deep seafloor that are new to computer vision. It features visually perplexing scenarios such as an octopus intertwined with a sea star, and confounding categories like vampire squids and sea spiders. This dataset can push forward research on topics like fine-grained transfer learning, novel category discovery, species distribution modeling, and carbon cycle analysis, all of which are important to the care and husbandry of our planet.

Related papers

• Reconstructing Animals and the Wild [51.98009864071166]
  We propose a method to reconstruct natural scenes from single images. We base our approach on advances leveraging the strong world priors in Large Language Models. We propose a synthetic dataset comprising one million images and thousands of assets.
  arXiv  Detail & Related papers  (2024-11-27T23:24:27Z)
• Does Spatial Cognition Emerge in Frontier Models? [56.47912101304053]
  We present SPACE, a benchmark that systematically evaluates spatial cognition in frontier models. Results suggest that contemporary frontier models fall short of the spatial intelligence of animals.
  arXiv  Detail & Related papers  (2024-10-09T01:41:49Z)
• Fish Tracking Challenge 2024: A Multi-Object Tracking Competition with Sweetfish Schooling Data [0.6834295298053009]
  The Fish Tracking Challenge 2024 introduces a multi-object tracking competition focused on the behaviors of schooling sweetfish. Using the SweetFish dataset, participants are tasked with developing advanced tracking models to accurately monitor the locations of 10 sweetfishes simultaneously. By leveraging video data and bounding box annotations, the competition aims to foster innovation in automatic detection and tracking algorithms.
  arXiv  Detail & Related papers  (2024-08-31T03:26:53Z)
• The FathomNet2023 Competition Dataset [0.8180522890142969]
  Ocean scientists have been collecting visual data to study marine organisms for decades. There are tools for automatically processing these data, but none that are capable of handling the extreme variability in sample populations. Models that are able to recognize when an image or video sequence contains a new organism, an unusual collection of animals, or is otherwise out-of-sample is critical to fully leverage visual data in the ocean.
  arXiv  Detail & Related papers  (2023-07-17T18:50:53Z)
• Context-Driven Detection of Invertebrate Species in Deep-Sea Video [11.38215488702246]
  We present a benchmark suite to train, validate, and test methods for temporally localizing four underwater substrates and 59 underwater invertebrate species. DUSIA currently includes over ten hours of footage across 25 videos captured in 1080p at 30 fps by an ROV. Some frames are annotated with precise bounding box locations for invertebrate species of interest.
  arXiv  Detail & Related papers  (2022-06-01T18:59:46Z)
• Unlocking the potential of deep learning for marine ecology: overview, applications, and outlook [8.3226670069051]
  This paper aims to bridge the gap between marine ecologists and computer scientists. We provide insight into popular deep learning approaches for ecological data analysis in plain language. We illustrate challenges and opportunities through established and emerging applications of deep learning to marine ecology.
  arXiv  Detail & Related papers  (2021-09-29T21:59:16Z)
• FathomNet: A global underwater image training set for enabling artificial intelligence in the ocean [0.0]
  Ocean-going platforms are integrating high-resolution camera feeds for observation and navigation, producing a deluge of visual data. Recent advances in machine learning enable fast, sophisticated analysis of visual data, but have had limited success in the oceanographic world. We will demonstrate how machine learning models trained on FathomNet data can be applied across different institutional video data.
  arXiv  Detail & Related papers  (2021-09-29T18:08:42Z)
• Cetacean Translation Initiative: a roadmap to deciphering the communication of sperm whales [97.41394631426678]
  Recent research showed the promise of machine learning tools for analyzing acoustic communication in nonhuman species. We outline the key elements required for the collection and processing of massive bioacoustic data of sperm whales. The technological capabilities developed are likely to yield cross-applications and advancements in broader communities investigating non-human communication and animal behavioral research.
  arXiv  Detail & Related papers  (2021-04-17T18:39:22Z)
• Generating Physically-Consistent Satellite Imagery for Climate Visualizations [53.61991820941501]
  We train a generative adversarial network to create synthetic satellite imagery of future flooding and reforestation events. A pure deep learning-based model can generate flood visualizations but hallucinates floods at locations that were not susceptible to flooding. We publish our code and dataset for segmentation guided image-to-image translation in Earth observation.
  arXiv  Detail & Related papers  (2021-04-10T15:00:15Z)
• What Can You Learn from Your Muscles? Learning Visual Representation from Human Interactions [50.435861435121915]
  We use human interaction and attention cues to investigate whether we can learn better representations compared to visual-only representations. Our experiments show that our "muscly-supervised" representation outperforms a visual-only state-of-the-art method MoCo.
  arXiv  Detail & Related papers  (2020-10-16T17:46:53Z)
• VisualEchoes: Spatial Image Representation Learning through Echolocation [97.23789910400387]
  Several animal species (e.g., bats, dolphins, and whales) and even visually impaired humans have the remarkable ability to perform echolocation. We propose a novel interaction-based representation learning framework that learns useful visual features via echolocation. Our work opens a new path for representation learning for embodied agents, where supervision comes from interacting with the physical world.
  arXiv  Detail & Related papers  (2020-05-04T16:16:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.